Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs
Authors: Yusheng Zhao
Qixin Zhang
Xiao Luo
Weizhi Zhang
Zhiping Xiao
Wei Ju
Philip S. Yu
Ming Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Text-Attributed Graphs
Large Language Models
Zero-Shot Inference
Primary area: applications
TL;DR: This paper tackles the problem of zero-shot inference on text-attributed graphs, and proposes a novel method that uses bundles to query LLMs and supervise GNNs.

Abstract: Large language models (LLMs) have been used in many zero-shot learning problems, with their strong generalization ability. Recently, adopting LLMs in text-attributed graphs (TAGs) has drawn increasing attention. However, the adoption of LLMs faces two major challenges: limited information on graph structure and unreliable responses. LLMs struggle with text attributes isolated from the graph topology. Worse still, they yield unreliable predictions due to both information insufficiency and the inherent weakness of LLMs (e.g., hallucination). Towards this end, this paper proposes a novel method named Dynamic Text Bundling Supervision (DENSE) that queries LLMs with bundles of texts to obtain bundle-level labels and uses these labels to supervise graph neural networks. Specifically, we sample a set of bundles, each containing a set of nodes with corresponding texts of close proximity. We then query LLMs with the bundled texts to obtain the label of each bundle. Subsequently, the bundle labels are used to supervise the optimization of graph neural networks, and the bundles are further refined to exclude noisy items. To justify our design, we also provide theoretical analysis of the proposed method. Extensive experiments across ten datasets validate the effectiveness of the proposed method.

Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs
Yusheng Zhao', Qixin Zhang', Xiao Luo";, Weizhi Zhang
Zhiping Xiao"; Wei Ju', Philipp S. Yu', Ming Zhang!* State Key Laboratory for Multimedia Information Processing,
School of Computer Science, PKU-Anker L.LM Lab, Peking University,
College of Computing and Data Science, Nanyang Technological University,
3 Paul G. Allen School of Computer Science and Engineering, University of Washington
Department of Computer Science, University of California, Los Angeles, Department of Computer Science, University of Illinois Chicago, yusheng. zhaoêstu. pku. edu. en, qixinzhang1 106raa/ 106@gmail com edu,
xiaoluodcs. ucla. patxiaoθux. edu, {juwei, mahang, cs) Ōpku edu.
Abstract
Large language models (LLMs) have been used in many zero-shot learning prob- lems, with their strong generalization ability. Recently, adopting LLMs in text- attributed graphs (TAGs) has drawn increasing attention. However, the adop- tion of LLMs faces two major challenges: limited information on graph struc- ture and unreliable responses. LLMs struggle with text attributes isolated from the graph topology. Worse still, they yield unreliable predictions due both information insufficiency and the inherent weakness of LLMs (e.g., hallucina- tion). Towards this end, this paper proposes novel method named Dynamic Text Bundling Supervision (DENSE) that queries LLMs with bundles of texts obtain bundle-level labels and uses these labels supervise graph neural net- works. Specifically, we sample a set of bundles, each containing a set of nodes with corresponding texts of close proximity. We then query LLMs with the bun- dled texts to obtaín the label of each bundle. Subsequently. the bundle labels are used to supervise the optimization of graph neural networks, and the bundles are further refined to exclude noisy items. To justify our design, we also provide theoretical analysis of the proposed method. Extensive experiments across ten datasets validate the effectiveness of the proposed method. Our code is available at https:// / thub. com hwt./nguueggg//////// bundoduunddeeeeeeeeeeeeeeeeeeeee neuree-nnnrpp22
Introduction
Text-attributed graphs (TAGs) [83, 87] are an important form of graph data, containing textual descriptions associated with each node. By combining textual information with non-Fuclidean graph topology. TAGs serve as natural structured data representations in many applications, including citation networks [67], social networks [61], e-commerce networks [48], and webpage networks [11]. As complete labeling of these large networks is often time-consuming and costly, efforts have been made utilize semi-supervised learning [33, 74, 51], transfer learning [12, 102, 95, 52], and few-shot/zero-sho learning [14, 91, 86] to understand text-altributed graphs with limited labels.
Large language models (LLMs) [84, 44] have been observed to exhibit strong zero-shot generalization capability, enhancing the performance on various types of data, including visual signals [97], audio
Conresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Tell me which category the following paper belongs to:
Individusl Query
Individual Supervision
Extracting Query Modifications from Nonlinear SVMs
label
Visual Contextual Awareness in Wearable Computiing
LT
label 2
Direct velue-approximation for factored MDPs
label 3
(o) Individual query and individual supervision
Tell the main category that most of the following papers belong to:
Query
Bundle Supervision
Extracting Query Modifications from Nonlinear SVMs
bundle
Context Page Analysis for Improved Web Search
label
Recommending Web Documents Based on User Preferences
(b) Bundle query and bundle supervision
Figure 1: (a) Querying LLMs with individual texts and supervising graph learning with individual
labels. (b) By creating text bundles, we perform bundle queries obtain bundle labels for supervision.
signals [92]. texts [45], programming code [85, 94], time series [93], and graphs [76]. Recently, there have been efforts in integrating LLMs in text-attributed graphs [20, 25, 101] for zero-shot inference. One line of research integrates the graph topology into language models [9, 53, 101], converting -un-luelddean topology into a sequence of tokens. However, building such foundation models requires large amount of data [77], and the conversion to Euclidean data inevitably incurs information loss [47]. Another line of research directly utilizes the zero-shot generalization ability of existing LLMs to understand node attributes [8, 38, 77]. and utilizes the output of LLMs as supervision signals for training graph neural networks (GNNs) [33] or as clustering centers [77]. However, the text attributes are often isolated from the graph topology, and the unreliable responses from LLMs also pose challenges for subsequent operations.
Adopting LLMs in zero-shot inference on TAGs faces two major challenges: (1) LLMs receive limited Euclidean, making difficult transform into token sequences with limited context windows. (2) LLMs yield unreliable responses. The inherent weakness of LLMs (e.g.. hallucination), together with limited information, makes the responses from LLMs unreliable, damaging subsequent operations like clustering, classification, or supervision.
information on graph structure. Graph topology
Towards this end, this paper proposes novel method named dynamic text bundling supervision that queries LLMs and supervises graph neural networks using text bundles. As is illustrated in Figure conventional methods [10, 77] query LLMs with individual text items (for example, in citation networks, this would be individual papers" titles and abstracts). The LLMs then return the annotations of these texts, which are used supervision signals. This paradigm faces the two major challenges mentioned above: the LLMs suffer from limited information, and the downstream supervision signals are unreliable. By comparison, this work proposes to query LLMs and supervise subsequent graph learning with text bundles. We first sample topologically or semantically similar text items to form text bundle, and then query the LLMs about the mode category (i.e., the most frequent category of the text items in the bundle) as the bundle label. Subsequently, we design bundle supervision that uses the bundle labels to train graph neural network, and during this process, bundles are further refined to exclude noisy items. In this way. the LLMs receive richer information from multiple interrelated text items in a bundle (challenge 1), and the predicted bundle labels are more robust to the uncertainty or misinterpretation of single text items with bundle supervision and refinement (challenge 2). We perform both theoretical and empirical studies to demonstrate the effectiveness of our method.
The contribution of this paper can be summarized as follows. We introduce a new perspective that connects bundle structure and text-attributed graphs provide robust supervision of graph neural networks. @ We propose a novel framework consisting of bundle sampling, bundle query, bundle supervision, and bundle refinement. We also provide rigorous theoretical analysis of our method, showing its tolerance to outlier nodes and the convergence properties of optimization. O We perform extensive experiments on ten aext-attributed graph datasets across various domains, and the results validate the effectiveness of the proposed method compared to competing basclines.

---

Text attributes T
Tell me the main category that most
supervise
Entropy Bosed Loss
Ranking Bosed Less
evicted
texts belong To:
4
bundle B
sompling
Text attribute
bundle lobel
GNN 9e
Text attribute
g
preserved
Text attribute
g
text arributed
graph (TAG)
g
(a) Bundle Sampling
(b) Bundle Query
(c) Bundle Supervision
(d) Bundle Refinement
The overall framework of method. We first sample nodes of proximity to form bundles (a), which are then used to query the LLM about their main categories (b). Subsequently, the bundle labels from the LLM's response are used to supervise graph neural network (c). During optimization, we further refine the bundle to exclude noisy nodes (d).
Figure 2:
2 Related Works
Text-Attributed Graphs. Text-attributed graphs (TAGs) are a special type of graphs whose nodes associated with textual attributes [83, 87]. They are common forms of data in many fields, such as citation networks [4], knowledge graphs [66], social networks [37], web page networks [19], etc. Research on TAGs generally focuses on combining textual attributes with graph structures, with the help of text embedding methods [75, 56] and network embedding methods [78, 42, 31]. As the annotation costs of TAGs are usually high, efforts have been made in semi-supervised learning [50, 90], transfer learning [89, 21, 102, 96], and few-shot learning [24, 91, 86]. With the advancement of large language models, this work makes step further, focusing the zero-shot inference of text-attributed graphs [10, 76, 77] with the help of L.MMs.
Large Language Model for Graphs. Large language models (LLMs) [84, 44] have shown impressive performance in understanding data beyond natural languages, including programming languages [28], sequences of numbers [29], mathematics [69], and graphs [10, 30, 65, 41]. LLMs exhibit strong generalization ability, enabling few-shot or zero-shot inference on graphs. One line of research aims to build foundation model, incorporating graph structures into current language model architectures [81, 16, 43, 76]. These methods often require training to align the graph structure and natural language [88, 102], involving large amount of labeled paired data. Another line of research makes use of the inference capability of existing LLMs to generate labels or related information of graphs [71, 10, 8, 77]. However, they often isolated nodes [77] explicit descriptions that hard for [.LMs to understand [72]. Additionally, the noisy labels generated by L.LMs can further harm subsequent inference operations (e.g., supervising neural networks, performing clustering) on graphs. Compared to these methods, this paper proposes to use text bundles to query LLMs and supervise graph neural networks, leading richer information and more robust optimization.
3 Methodology
Problem Definition. We denote a text-attributed graph as 9 EV.ET. where V is the set of nodes, E the set of edges. T is the set of textual attributes, and y is the set of node labels. Each node U; E V is associated with textual descriptions 4 E T and the corresponding label ys. For each node, we can obtain its vectorized embedding via a text encoder fe, i.e., æ; fo(4,) E Rd. We denote the total number of nodes as [V]. A node bundle is defined and a text bundle corresponding to the node bundle is defined as set of text attributes associated with the node bundle. For simplicity, of corresponding node bundles and text bundles. The goal of zero-shot inference on attributd graphs is to infer the node labels )' according to the graph topology V. E, and the textual attributes T.
set of nodes in the graph,
use the term bundle and notation B to denote the indices
3.1 Framework Overview
The overall framework of the proposed method illustrated in Figure We first perform bundle sampling, constructing node bundles according to topological or semantic proximity (Section 3.2). With the obtained node bundles, we transform the corresponding text bundles into prompts and query the LLM about the most frequent category of the bundle (Section 3.3). With these bundle labels,
3

---

we perform bundle supervision, training graph neural networks with cntropy-based and ranking- based supervision. Additionally, theoretical analysis is provided regarding the properties bundle supervision to justify our design (Section 3.4). During the optimization process, we further refine the bundles dynamically to exclude noisy components (Section 3.5).
3.2
Bundle Sampling
We first introduce the method for sampling bundles. Intuitively, we aim for most nodes within bundle belong to the same category (i.e. strong mode), so that LLMs more easily predict the mode category and the bundle label more accurately reflects the nodes it contains. To achieve this, we sample nodes of close proximity. Specifically, we first randomly sample the core node ve from the set of nodes V. and then sample the rest of the nodes. We fix the size of a bundle n, and design two criteria for sampling: topological proximity and semantic proximity.
Topological Proximity. For a given core node Ve in graph G, a common assumption is that node similar to nodes topologically close to itself [33, 18]. Formally, given two nodes de and v, their topological proximity can be measured by the length of the shortest path from and denoted as dd (Ve, v). and we can define topologically similar nodes with respect vee as:
NY (v.) {i fillss d (1;, vo) S k). k = =in fo (V*((.)) 2 uB 1}
where & is an adaptive hop size. For core nodes with many (k-hop) neighbors, a smaller hop size is used, and vice versa. We then sample (ng 1) nodes from the neighborhood NV (ve) to form the bundle B together with the original core node V...
Semantic Proximity. For graphs with heterophily, topological proximity hardly entails similarity [100, 99, 98]. Therefore, we turn semantic proximity utilizing vectorized representations of nodes. Specifically, given embeddings of each node X' embedding Ze. we construct the node bundle based on the closeness in the embedding space Rd:
{ and core node v, with corresponding
B
{i 2;
E N (x=)]:
where N (a:.)) denotes the set of top ng vectors X' that are closest to Te in terms of Fuclidean distance (i.e., Lg distance) in the embedding space.
In practice, different criteria are adopted for different types of graphs. For graphs with high homophily (e.g., citation networks), topological proximity is used. For graphs with high heterophily (e.g., webpage networks), semantic proximity adopted. We repeatedly sample a of node bundles as { [B1, B2.... Bng }, where is the number of bundles. For simplicity, we omit the subscript of bundles and use B for an arbitrary bundle in the following discussions.
3.3 Bundle Query
We then query LLMs to obtain information about the bundles. While it might be straightforward to provide individual text attributes for node-level pseudo-labels, this approach carries the risk of limited information (as the LLMs only receive information from single isolated node attribute) and unreliable responses (since the output pseudo-labele can be highly noisy). By using bundling, LLMs receive more information from proximate nodes, making the decision regarding the mode category easier than individual classification, which results more reliable annotations.
With the node bundles selected, we obtain their corresponding text bundles and construct a single prompt P(B) for each text bundle with dataset description and task description:
P(B)= (dataset_ caataset_descriptionn Concat (t.li E (B))(tas__eescripiin)) description).
(3)
where the Concat(-) operator concatenates all the text attributes in the bundle. We then query the LLM with the prompts to obtain the mode category of the bundle, denoted as T3.
3.4 Bundle Supervision
The bundle labels are then used to supervise graph neural network. Since bundle label represents the mode category that most nodes in the bundle belong to, nodes from other categories may also be included. Therefore, effective bundle supervision requires tolerance for these "outliers". To address

---

this, we design two supervisions: antropy-based supervision and ranking-basee supervision. We denote the graph neural network as go, and generates probability distributions for each node as:
{ go ((o:)i=r.)) P: softmax(z:),
(4)
where 2; E C is the logits, P: E CC is the probability, and C' is the number of classes.
Entropy-based Supervision. When bundle B has label j the nodes in it are likely to fall into class j on average. Therefore, we compute the bundle class distribution p(B) and the corresponding bundle-level entropy- objective function LB as follows:
P( = softmax
LBE = CE (p(B),
(5)
B
iEB
where CE(,,)) .) is the cross-entropy loss. We then theoretically demonstrate that this bundle supervi- sion (i.e., LBB) is more tolerant to outliers compared to individual supervision using cross-entrope Formally, we have the following theorem (with the proof in Appendix A):
Theorem 3.1. Given a bundle B, its corresponding bundle class distribution p(B3 = P2I.P2.... .p)). an outlier node Vo. O E B with probability distribution po (p'1 ......, Pc). denote m
argmax,/v/Jie If the bundle label I + m', and p'm 2 pm', we have:
ÖCBE
θLIE_ Ologr'm. where LBZ= CF. (p(B), and LIE
1 B
0 S Flo
CE (p.: 1),
(6)
where ÿ is the bundle label, LB is bundle supervision and Lie is individual supervision.
Remark 1. Theorem 3.1 suggests that when encountering "outlier" nodes that conftict with the predicted mode category and the bundle distribution (i.e., the condition F m' and p 2 Pm' in the theorem), the bundle cross-entropy objective function (i.e., LBE defined in Eq. 5) is more tolerans compared supervising the nodes in the bundle individually (i.e.. LI defined in the theorem), as evidenced by a smaller penalty imposed by the gradient.
beased Supervision. To ensure that the supervision focuses more bundles where the predicted bundle labels do not dominate the bundle's bundle probability distribution, we adopt the concept of ranking loss [7, 57, 79], and design a ranking- based loss as follows:
CR --in
C min (log ((B) log i=1 (p(B).}.0)
(7)
where p(B); R denotes the i-th component of vector p(B) (i.e., the predicted probability of class i). When the category of the bundle label ه is not the highest in the predicted bundle probability distribution by the GNN go, the bundle B is penalized by this loss function. On the other hand, when the category of gd has high bundle probability p(B) (which does not necessitate all z,, E B to be high), the loss will be zero. In our implementation, a combination of the two supervision objectives is used, leading to the final objective function as follows:
C = LB + CR-
(8)
Theoretical Analysis. We then aim to present rigorous theoretical analysis of the proposed method, focusing in particular on the convergence of the bundle supervision process. Before going into the details, first examine the smoothness of our entropy-hasedd objective LB.- More specifically, have the following results (with the proof in Appendix B):
Theorem 3.2. Given a graph neural network ga. if its corresponding first-order and second-order partial derivatives are bounded, that is, (V,,, I G and max( V (0)l) S M where 2i, the "c'-th logit of the outpul vector 2; A (zi,1, 2i,c) provided by GNN go. then we can show that the cross-entropy loss function LA (0) defined in Eq. 5 satisfies the following conditions:
i): The cross-entropy loss fanction (BE has bounded gradient, i.e., [[ S where the symbol [B] represents the cardinality of bundle B:
20
T8]
ii): The second-order partial derivatives of the cross-entropy loss function LBE(O) is also bounded, namely, max (1V (0)( < 2(M+G)) which simultaneously means the loss
5

---

CBE(O) ( B
-smooth, that is,
2na(M + G²)
(V - VLLE((2))) S
|00 -82112.
[B]
where na is the dimension of the unknown parameter 8.
Remark 2. worth noting that, Theorem 3.2, the symbol max (M represents the maximum absolute value among the elements of matrix M. Moreover, || and || ll denote the standard L2 norm and Los nor,, respectively.
Remark 3. Theorem 3.2 indicates that the smoothness and differentiability of the graph neural network ga can, to some extent, be inherited by our adopted cross-entropy loss function LBE-
With the results of Theorem 3.2, we next show that, under some mild conditions, the commonly used gradient descent algorithm for training GNN can finally converge to stationary point of our adopted loss function L BE + LR). Before that, we first characterize the dynamics of the general gradient descent algorithm, namely, we suppose f-11 - 0, V R where 1) > 0 is the learning rate and the time E 21. TY. Subsequently, we present the detailed results regarding the convergence of our adopted bundle supervision process, that is,
Theorem 3.3 (Proof is deferred to Appendix C). Under the assumptions of Theorem 3.2 and the condition 7) < " if. when the iteration index t is large, the model parameter 9 provided by gradient descent algorithu can effectively fit the predicted bundle label i" namely, g" arg max,e(),.... {pe, (B)s), then we can verify that the final obtained model parameter 0r+1 will converge to a stationary point of the adopted loss function L(O),, that say. [VL(0++))l can approach toward small value as? T + Co.
B
Remark 4. The proof of Theorem 3.3 builds upon the standard non-convex optimization frame- works [27, 2, 36]. Moreover, it is important to emphasize that when GNN model ge possesses certain structural properties, extensive research has shown that the resulting stationary point of the aforementioned gradient descent algorithm exhibit strong generalization capabilities [35, 5] and in some cases, may even correspond to a global minimum [70, 59, 55].
3.5 Bundle Refinement
During optimization of the graph neural network Jø. the bundle B may include nodes that do not belong to the category of g To address this, we design the bundle refinement process that excludes these noisy nodes by evicting those with lower confidence in class jن Specifically, given the node- level probability distribution in a bundle, i.e., Pt, i B, we denote the confidence of p; with respect to class ÿ" as P..a. We evict the less confident node in the bundle
B (liiEB Ps,s
min jEB
Pj.g
where denotes the update of the bundle. Bundle refinement is performed multiple times during the optimization process of ga. By evicting the less confident nodes that are potentially misaligned with the bundle label, the noise in bundle supervision is further reduced. Through bundle refinement, the initial bundles, sampled via topological or semantic proximity, are dynamically adjusted during the supervision of the graph neural network fulfill the predicted bundle label j queried from the LLM, making the proposed method robust. The overall algorithm is provided in Appendix D.
4 Experiments
4.1 Experimental Setup
Datasets. In the experiments, we use ten representative datasets, i.e., Cora [54], CiteSeer [17]. Wikies [58], History [62], Children [62], Sportsfit [62], Cornell [11], Texas [11], Wisconsin [11], and Washington [11]. Among these datasets, Cora and CiteSeer are citation networks. Wikics is
knowledge graph derived from Wikipedia. History, Children, and Sportsfit are e-commerce networks
of different types of products (i.e., history books, children's literature, sports goods). Cornell,

---

Texas, Wisconsin, and Washington are web page networks of universities. The datasets cover both homophilic and heterophilic graphs, with the first six datasets of high homophily and the last four of low homophily. More details about the datasets can be found in Appendix E.
Compared Baselines. We compare spectrum of methods with method. The compared methods include the following categories. > Text encoders, including SBERT [68], RoBERTa [49], OpenAl's Sext-Embeodiing--Laee
(TE-3-Large) [63] and LLM2Vec [3]. Generative LLMs, including GPT-3.5-turbo [1] and GPT-4o [26]. Graph self-supervised learning methods, including DGI [73]
and GraphMAE [23]. & Graph foundation models or graph learning methods with LLMs, including
OFA [46]. GOFA [34]. UniGLM [15], ZeroG [40], GraphGPT [72], LLAGA [6], and LLM-BP [77].
More details about the baseline methods can be found in Appendix F.
Implementation Details. In the experiments, we use GPT-4o [26] as the default L.LM for bundle query. In bundle sampling, we set the bundle size n as 5 and the number of bundles ng as 100 for all datasets. For homophilic graphs, we use topological proximity for sampling and GCN [33] as the default GNN, whereas semantic proximity and GloGNN [39] are used in heterophilic graphs. We train the GNN on an NVIDIA RTX 3090 GPU for 500 epochs, and bundle refinement is performed at the 300th and 400th epochs. More details of our implementation can be found in Appendix G.
Table 1: Prediction accuracies of our method compared to baselines across datasets. We mark the best results in bold and the second-best with underline.
Method Cora
CiteSeer
WikiCS
History
Children
Sporisfit
Comell
Texas
Wisc.
Wash.
SBERT
69.75
66.69
59.06
53.53
22.59
43.79
63.66
64.58
62.10
63.52
RoBERTa
70.71
66.95
59.08
55.39
24.25
41.51
61.68
62.25
60.33
60.60
TE-3-Large
71.90
66.24
61.78
50.15
24.68
58.39
81.50
75.42
73.14
66.35
LLM2Vec
67.34
67.13
62.34
53.14
25.56
57.00
81.26
76.68
73.36
65.92
GPT-3.5-turb
70.11
66.83
65.53
55.07
29.73
67.21
45.54
56.14
58.86
51.09
GPT-4o
70.29
64.77
66.10
53.30
30.76
66.35
45.54
63.10
56.60
48.90
DGI
16.79
15.24
14.98
20.98
2.22
7.48
14.66
11.23
12.08
20.96
GraphMAE
15.13
8.11
8.91
36.36
7.24
30.50
23.04
17.65
23.02
24.89
OFA
20.36
41.31
30.77
8.25
3.05
15.18
29.84
11.77
4.80
6.04
GOFA
71.06
65.72
68.62
56.25
12.15
37.87
39.50
38.37
32.51
31.02
UniGLM
45.57
52.26
55.05
44.24
21.48
33.46
23.03
21.39
27.16
24.01
ZeroG
60.40
50.35
46.74
36.55
12.72
14.27
10.47
53.48
12.66
8.30
GraphGPT
17.48
13.93
33.59
12.31
9.94
4.53
10.18
18.48
12.35
20.64
LLAGA
11.62
19.52
10.98
7.95
10.09
1.84
12.57
15.51
15.09
10.48
LLM-BP
72.59
69.51
67.75
59.86
24.81
61.92
83,28
81.66
77.75
73.14
DENSE (ours)
75.09
72.37
71.03
67.31
31.75
75.88
84,82
92.51
87.17
81.66
4.2 Main Results
Comparison with Existing Methods. We compare our method against 15 baselines across 10 datasets in Table 1. From the results, we can see that our method consistently outperforms competitive baselines all 10 datasets, showing the effectiveness of the proposed text bundling method. Text embedding methods (e.g.. SBERT, LLM2Vec) and generative LLMs (e.g., GPT-4o) achieve moderate performance on many datasets. However, their ignorance of the graph topology leads to weaker performance, especially when the structures are important. Graph self- self-supervised learning methods (egg. DGI, GraphMAE) generally yield low accuracy without the assistance of LLMs and their strong generalization capability. For foundation models (e.g., GOFA, ZeroG) that incorporate graphs in LLMs for joint training, their high performance is not consistent, worsening with graphs out of their original training distribution (e.g.. in university web page networks). By comparison, our method consistently outperforms basclines on various datasets covering different domains. Additionally, our method is agnostic to the specific architecture of the graph neural network, allowing us to flexibly benefit from the advancement of GNN architectures when facing different types of graph structures (e.g., homophilic graphs and heterophilic graphs).
Performance Under Different LLM Backbones. We also show the prediction accuracies of our method using different LLMs. Specifically, we provide results on five LLMs, including GPT-4o [26] (used as default), GPT-335-turbo [1], 1PPT-4.1-nane [64], Deepseek- V3 [44], and Gemini-2.5-flann [13]. The results on four datasets (i.e,, Cora, History, Sportsfit, Texas) are shown in Table 2. As can be seen from the results, using alternative I.LMs generally yields satisfactory performance on average. Among these LLMs, GPT-4o and Gemini-2.5 2.5llash perform relatively better, while cheaper
7

---

Table 2: The prediction accuracies under different LLM backbones on four datasets. The best is marked in bold and the second-bes underline.
Table 3: Ablation studies on four datasets.
Method
Cora
History
Sportsft
Texas
VI: R.S.
70,48
61.80
65.60
88.24
LLM
Cora
History
Sportsfit
Texas
V2: IQ.
71.96
63.95
72.61
84.49
GPT-4o 75.09
67.31
75.88
92.51
V3: who Cae
70.11
65.29
91.44
GPT-3.5-turbo
73.25
69.87
89.30
V4: wo Ln
73.99
66.73
75.48
86.10
GPT-4.1-mam
71.09
90.37
V5: w/ Lie
73.43
66.29
74.05
85.03
Deepscek-V3
75.28
67.00
73.52
85.56
V6: w/o B.R. 73,89
66.55
73,00
91.98
Gemini-2.5
73.25
20.08
74.98
93.05
DENSE (ours)
75.09
67.31
75.88
92.51
or older LLMs like GPT-3.5-turbo, GPT-4, --nao, Deepseek- V3 yield decent accuracies well. This suggests that our method can benefit from the advancement of LLMs.
4.3 Ablation Studies
We investigate how the different mechanisms used our method affect the final accuracy, and we present the ablation studies in Table 3. We construct of variants of our method (marked V1 to V6): VI uses random sampling (R.S.) instead of topological proximity or semantic proximity to obtain bundles. V2 uses individual query (I.Q.), asking the LLM about the category of each node with the text attribute. V3 removes the entropy-based loss LBg. V4 removes the lanking-based loss CR. V5 uses individual supervision, i.e., Lıg defined in Theroem 3.1. V6 does not employ bundle refinement. As can be seen from the results, each technique proposed is helpful for the overall accuracy, and removing them causes performance degradation. Additionally, we find that bundle sampling is important, especially when the number of classes is large (in this case, 12 classes for History and 13 classes for Sportsfit, both of which witness severe drop in accuracy with random bundle sampling). One explanation is that inappropriate sampling causes the nodes in bundle to be more uniformly distributed across various categories, making it difficult to decide the bundle class (with weaker mode category) and perform bundle supervision (with noisier bundle labels). Moreover, we find that individual supervision (Cre) weaker than bundle supervision, which suggests that our supervision method is more tolerant to bundle outliers.
L.Q.
B.Q.
75
Accuraey (%)
72
Acguracy (36
722
Aecurat (%6
75
69
69
66
66
25
63
Cora
History
63
History
Cora Citeseer
Wikcs
Histary
Children
Spurtefft
Carnell
Tous
Wise.
Wash.
100 150
200
Bundle Size
Number of Bundles
Figure 3: Left: prediction accuracies under different bundle sizes (i.e., Rg)). Middle: prediction accuracies with different numbers of bundles (i.e., ns)). Right: accuracy comparison of individual query (I.Q.), bundle query (B.Q.), and our method (Ours).
4.4 Hyperparameter Analysis
Effects of Bundle Size nn. We also show the prediction accuracies using different bundle sizes n in Figure 3 (Left). As can be seen from the figure, setting ng to achieves relatively good performance. Smaller bundle sizes make it difficult to provide sufficient information about the neighborhood, while larger bundle sizes have the risk of including categories other than the mode category. Moreover, we observe that odd bundle sizes are relatively better than even ones. A possible explanation is that even bundles are more likely to have ties (e.g., two nodes from class A and two from class B in a four-node bundle), causing confusion and noisy supervision. This is less significant in datasets with larger number classes (e.g., g., the History dataset with 12 classes), where odd bundles may also include a node from the third class, failing resolve the ties.
Efffett of the Number of Bundles nts. We also show the prediction accuracies with different numbers of bundles in Figure 3 (Middle). As can be seen from the figure, larger numbers of bundles generally
8

---

(a) Prompt Template:
(b) Prompt Exumple (CiteSeer):
have <node_ description> in
We have description or opening text of scientifie publications in a citation network from the following categories: I' l'Agents', 'machine (ea))in(MML), 'information retrieval (IR), 'databuse (DB), 'human- computer interaction (HCI), 'artificial intelligence (AI)] Below are texts from papers.
<graph description> from the following
<number of classes> categories: [<oateçory 1>,
cousegury 2?... <category [>] Below are texts from <bundle__t xe> <node_ refer_ name ploral>. [<node refer name_ singular> 1]
[Paper 1]
<text_ fecxt_attribute of rade_ 1>
Extracting Query Modifications from Nonlinear SVMs...
I<node zefer name singular> 2|
[Paper 2]
<text_ Sccxt_attribute___ of rude_ 2>>
Context and Page Analysis for Improved Web Search...
Please tell me the main category that most of the papers belong to. Think carefully and then provide the final answer in the last line, using the format: "Final Answer: <category>". "
Please tell me the main category that most of the papers belong to. Think carefully and then provide the final answer the last line, using the format: "Final Answer: <category>",
(c) LLM's Response (CiteSeer, GPT-40):
To determine the main category that most of the papers belong to, let's analyze the content of e'acch paper and try to classify according to the given categories:
Paper Extracting Query Mudifications from Nonlinear SVMs. The focus is query mudifications and using support vector machines (SVMs) to improve precision and recall in search. SVMs are common tool in machine learning, and the paper discusses dimensionality reduction and feature space, which fit within the machine learning domain, also talks about query modifications which slightly implies information retrieval (IR). Paper 2: Context and Page Analysis for Improved Web Search. Discusses the development a metasearch engine to improve web search efficiency and precision by analyzing query contexts, The main focus is on handling web search limitations which aligns with information retrieval (IR). Paper Paper 4... Paper (detuils omitted for paper 3.4, and 5)
Analyzing all papers, while machine learning techniques are frequently used, the persistent theme across most papers revolves around improving scarch engines, query modifications, and personalized search, which are strongly indicative of the "information retrieval (IR)" category.
Final Answer: information retrieval (IR)
The prompt template of bundle query (a), example of the prompt on the CiteSeer dataset (b), and an example of the response of GPT-4o to the query (c).
Figure 4:
yield better accuracy. As one bundle corresponds to one query of the LLM,, increasing the number of bundles indicates additional feedback from the LLM,, leading to additional supervision signals that better cover the graph topology and bring robustness from redundancy. Nevertheless, additional queries incur additional computation resources. Balancing performance and cost from querying LLMs, we set the number of bundles ns to 100 in the experiments.
4.5 Further Analysis
Bundle Query v.s. Individual Query. We then show that the LLM can better handle bundle queries than individual queries. Specifically, we provide the LLM's classification accuracies given individual queries (marked as I.Q.) and bundle queries (marked as B.Q.) in Figure 3 (Right). From the figure, we can see that predicting the main category of the text bundles is generally easier than classifying individual text items, and in some datasets (e.g., CiteSeer, Cornell), the improvement is fairly large. We also show the overall prediction accuracies of our method, and we can see general connection between the improvement of bundle queries and our method (compared to individual queries). This shows that the proposed text bundling method increases the reliability and robustness of supervision signals from LLMs and thereby improves the overall performance.
Prompt Examples and the LLM's Response. We also provide the prompt template, an example of the prompt, and the LLM's response in Figure 4. In the prompt, we provide information about the nodes and the graphs. We then ask the LLM to find the main category that most of the papers in the text bundle belong to. For the LLM's response, we can see that although machine learning is a frequent topic of research among the papers in the bundle, the LLM discovers a "persistent theme across most papers" to be strongly related to information retrieval. Without text bundling, the LLM may hesitate between machine learning and information retrieval when classifying Paper 1, as its analysis suggests that this paper "fits within the machine learning domain" and also "slightly implies information retrieval". Such ambiguity would cause noise in classification results and be harmful for potential subsequent operations (e.g., clustering, supervision of GNNs). By comparison, our method

---

allows the LLM to obtain more information, finding persistent theme that represents most text items in the bundle, improving the reliability of LLM's response.
Discussions
Decision of Graph Homophily. In Section 3.2, we provide two approaches for bundle sampling: topological proximity and semantic proximity, depending on the graph homophily. In practice, when homophily is not obvious from the graph's 's metadata, it is non con-trivial to determine which sampling introduce approach initially proposed by Wang al. [77] that
technique to adopt. Therefore, estimates the graph homophily by querying the LLM with text pairs of adjacent nodes. The LLM is asked to determine whether they belong to the same category. The ratio of positive response (i.e., "Yes") can be used as an indicator of graph homophily, In their paper, Wang et al. [77] show that it is possible to effectively approximate the homophily degree with limited queries of less powerful LLMs (e.g., GPTT-4-miini. Other potentially useful techniques include computing the cosine similarity of feature pairs from adjacent nodes instead of querying LLMs.
Potential Extension to Non-Text-Attributee Graphs. The proposed framework can be extended to general setting where text attributes are not provided. Non-text-attribun graphs can be converted to text-attributed graphs, which have been explored by Wang et al. [80]. By augmenting non-text-attribuun graphs with textual descriptions, we can apply the proposed DENSE framework for zero-shot inference on these graphs.
Bundle Refinement Configurations. In Section 3.5 and Section 4.1, we mention that the bundle refinement performed at the 300th and 400th epochs, evicting one item each time. We make a further explanation of these heuristic values: want to refine the bundle multiple times, and as the optimal bundle size is 5 (as empirically demonstrated in Section 4.4), evicting one item seems a reasonable choice (evicting more than item will significantly reduce the richness of information in the bundle). As for the time of refinement, we observe that the model generally converges at the 300th epoch, and after refinement, it usually converges within another 100 epochs.
Limitations. This paper focuses on text-attributed graphs, where each node associated with a textual attribute. For graphs where node attributes are hard for LLMs to understand, the proposed text bundling method is not directly applicable. For graph structures on which GNNs are inherently weak inferior to alternatives, this method may not be directly applicable.
Broader Impacts. As for broader impacts, the proposed text bundling method improves the zero- shot inference ability of LLMs on text-attributed graphs, facilitating downstream applications in many fields, including social network analysis, recommendation systems, web page analysis, and knowledge graph understanding.
6 Conclusion
This paper investigates the important problem of zero-shot inference text-attributed graphs with the help of LLMs. While previous efforts suffer from limited information on graph structure and unreliable responses, this paper proposes novel method named dynamic text bundling supervision that queries the LLM with text bundles to obtain bundle-level labels. Subsequently, the bundle labels used to supervise graph neural network, which is then used for classification. We provide theoretical analysis of our method, showing its tolerance of outlier nodes in the bundle and the convergence properties of optimization. We further refine the nodes in the bundle exclude noisy items. Extensive experiments are performed on ten datasets across different domains against number of competing baselines, and the results confirm the effectiveness of the proposed method.
Acknowledgement
Ming Zhang and Yusheng Zhao are supported by grants from the National Key Research and Development Program of China with Grant No. 2023YFC3341 and the National Natural Science Foundation of China (NSFC Grant Number 62276002). The authors are grateful to the anonymous reviewers for critically reading this article and for giving important suggestions to improve this article.
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint MxXi::3333008774 2023.
[2] Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via over-paramettrrizaton. International conference on machine learning, pages 242-252. PMLR, 2019.
[3] Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy, Llm2vec: Large language models are secretly powerful text encoders, arXiv preprint arXiv:2404.05990 2024.
[4] Dan Berrebbi, Nicolas Huynh, and Oana Balalau. Graphcite: Citation intent classification in scientific publications via graph embeddings. In Companion proceedings of the web conference 2022, pages 779-783, 2022.
[5] Thijs Bos and Johannes Schmidt-Hieber. Convergence rates of deep relu networks for multi- class classification. Electronic Journal of Statistics, 16(1):2724-2773, 2022.
[6] Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant. arXiv preprint arXiv:2402.08/ 01.02222208//70 70, 2024.
[7] Wei Chen, Tie- Yan Liu, Yanyan Lan, Zhi-Ming Ma. and Hang Li. Ranking measures and loss functions in learning to rank. Advances in Neural Information Processing Systems, 22, 2009.
[8] Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, et Exploring the potential of large language models (IIms) in learning on graphs. ACM SIGKDD Explorations Newsletter, 25(2):42 61, 2024.
[9] Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song, Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, et al. Text-space graph foundation models: Comprehensive benchmarks and new insights. arXiv preprint arXiv:2406. /0727, 2024.
[10] Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, and Jiliang Tang. Label-free node classification graphs with large language models (Ilms). arXiv preprint arXiv:2310. 046668 2023.
[11] Mark Craven, Dan DiPasquo, Dayne Freitag, Andrew McCallum, Tom Mitchell, Kamal Nigam, and Seán Slattery. Learning to extract symbolic knowledge from the world wide web. AAAI/AAI,, 3(36))2, 1998.
[12] Quanyu Dai, Xiao-Ming Wu, Jiaren Xiao, Xiao Shen, and Dan Wang. Graph transfer learning via adversarial domain adaptation with graph convolution. IEEE Transactions Knowledge and Data Engineering, 35(5):4908-4922, 2022.
[13] Google DeepMind. Gemini 2.5 flash model. gemini/#gemini- 25, 2024.
[14] Kaize Ding, Jianling Wang. Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. Graph prototypical networks for few-shot learning on attributed networks. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 295-304, 2020.
[15] Yi Fang, Dongzhe Fan, Sirui Ding, Ninghao Liu, and Qiaoyu Tan. Uniglm: Training one unified language model for text-attributed graphs arXiv preprint arXiv:2406.1205 2024.
[16] Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. Talk like a graph: Encoding graphs for large language models. arXiv preprint 23Xiv:2310004550 2023.
[17] C Lee Giles, Kurt Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries, pages 89-98, 1998.

---

[18] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855 855-864, 2016.
[19] Tao Guo and Baojiang Cui. Web page classification based on graph neural network. In International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing. pages 188-198. Springer, 2021.
[20] Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Harnessing explanations: LIm-to-lm interpreter for enhanced text-attributed graph representa- tion learning. arXiv preprint arXiv:2305.1 /9523, 2023.
[21] Yufei He, Yuan Sui, Xiaoxin He, and Bryan Hooi. Unigraph: Learning a unified cross- croms-domain foundation model for text-attributed graphs. arXiv preprint arXiv:2402.136300 2024.
[22] Roger A Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
[23] Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang. Chunjie Wang, and Jie Tang. Graphmae: supf-supe masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 594-604, 2022.
[24] Xuanwen Huang, Kaiqiao Han, Dezheng Bao, Quanjin Tao, Zhisheng Zhang, Yang Yang, and
Qi Zhu. proppt-baeed node feature extractor for few-shot learning on text-attributed graphs. arXiv preprint arXiv:2309.0288 2023.
[25] Xuanwen Huang, Kaiqiao Han, Yang Yang, Dezheng Bao, Quanjin Tao, Ziwei Chai, and Qi Zhu. Gnns as adapters for ilms on text-attributed graphs. In The Web Conference 2024, 2024.
[26] Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
[27] Prateek Jain, Purushottam Kar, et al. Non-convex optimization for machine learning, Founda- tions and Trends@ in Machine Learning, 10(3-4):142333,, 2017.
[28] Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. Self-planning code generation with large language models. ACM Transactions on Software Engineering and Methodology. 33(7):1-30, 2024.
[29] Yushan Jiang, Zijie Pan, Xikun Zhang, Sahil Garg, Anderson Schneider, Yuriy Nevmyvaka and Dongjin Song. Empowering time series analysis with large language models: A survey. arXiv preprint arXiv:2402.03188. 2024.
[30] Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han. Large language models graphs: A comprehensive survey. IEEE Transactions on Knowledge and Data Engineering, 2024.
[31] Wei Ju, Zheng Fang, Yiyang Gu, Zequn Liu, Qingqing Long, Ziyue Qiao, Yifang Qin, Jianhao Shen, Fang Sun, Zhiping Xiao, al. A comprehensive survey on deep graph representation learning. Neural Networks, page 106207, 2024.
[32] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1xXiv:1412.6980 2014.
[33] Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. arXiv preprint 16Xiv:1609.0290) 2016.
[34] Lecheng Kong, Jiarui Feng, Hao Liu, Chengsong Huang, Jiaxin Huang, Yixin Chen, and Muhan Zhang. Gofa: A generative one-for-all model for joint graph language modeling, arXiv preprint arXiv:2407.00700 2024.
[35] Sanjukta Krishnagopal and Luana Ruiz. Graph neural tangent kernel: Convergence on large graphs. In International Conference on Machine Learning, pages 17827-17841. PMLR, 2023,
12

---

[36] Guanghui Lan. First-order and stochastic optimization methods for machine learning. vol- ume Springer, 2020.
[37] Quan Li, Xiaoting Li, Lingwei Chen, and Dinghao Wu. Distilling knowledge on text graph for social media attribute inference. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2024-2028, 2022.
[38] Rui Li, Jiwei Li, Jiawei Han, and Guoyin Wang. Similarity-basee neighbor selection for graph llms. arXiv preprint arXiv:2402.03720, 2024.
[39] Xiang Li, Renyu Zhu, Yao Cheng, Caihua Shan, Siqiang Luo, Dongsheng Li, and Weining Qian. Finding global homophily in graph neural networks when meeting heterophily. In International Conference on Machine Learning, pages 13242-13256. PMLR, 2022.
[40] Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li. Zerog: Investigating cross- dataset zero-shot transferability graphs. Proceedings of the ence on Knowledge Discovery and Data Mining, pages 1725-1735, 2024.
the 30th ACM SIGKDD Confer-
[41] Yuhan Li, Peisong Wang, Xiao Zhu, Aochuan Chen, Haiyun Jiang, Deng Cai, Victor W Chan, and Jia Li. Glbench: A comprehensive benchmark for graph with large language models. Advances in Neural Information Processing Systems, 37:42349-42368, 2024.
[42] Lizi Liao, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. Attributed social network embedding. IEEE Transactions Knowledge and Data Engineering, 30(12):2257-2770 2018.
[43] Tianqianjin Lin, Pengwei Yan, Kaisong Song, Zhuoren Jiang, Yangyang Kang, Jun Lin, Weikang Yuan, Junjie Cao, Changlong Sun, and Xiaozhong Liu. Langgfm: A large language model alone can be a powerful graph foundation model. arXiv preprint 0rXiv:2410.14900 2024.
[44] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseck-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
[45] Han Liu, Siyang Zhao, Xiaotong Zhang, Feng Zhang, Wei Wang, Fenglong Ma, Hongyang Chen, Hong Yu, and Xianchao Zhang. Liberating seen classes: boosting few-shot and zero-shot text classification via anchor generation and classification reframing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18644-18652 2024.
[46] Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. arXiv preprint 2rXiv:2310.00149 2023.
[47] Jiawei Liu, Cheng Yang. Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang. Lichao Sun, Philip S Yu, et al. Graph foundation models: Concepts, opportunities and challenges. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[48] Weiwen Liu, Yin Zhang, Jianling Wang, Yun He, James Caverlee, Patrick PK Chan, Daniel Yeung, and Pheng-Am Ann Heng. Item relationship graph neural networks for e-commerce. IEEE Transactions on Neural Networks and Learning Systems, 33855:785-4479, 2021.
[49] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: /692, 2019.
[50] Xiao Luo, Yusheng Zhao, Zhengyang Mao, Yifang Qin, Wei Ju, Ming Zhang, and Yizhou Sun. Rignn: A rationale perspective for semi-supervised open-world graph classification. Transactions on Machine Learning Research, 2023.
[51] Xiao Luo, Yusheng Zhao, Yifang Qin, Wei Ju, and Ming Zhang Towards semi-supervised universal graph classification. IEEE Transactions on Knowledge and Data Engineering, 36(1):416-428, 2024.
13

---

[52] Xin Ma, Yifan Wang, Siyu Yi, Wei Ju. Bei Wu, Ziyue Qiao, Chenwei Tang, and Jiancheng Lv. Pala: Class-imbalanced graph domain adaptation via crototype-anchoree learning and alignment. In Proceedings of the Thirty-Fourd International Joini Conference on Artificial Intelligence, pages 1198-3207, 2025.
[53] Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma, Tong Zhao, Neil Shah, Mikhail Galkin, and Jiliang Tang. Position: Graph foundation models are already here. In Forty-first International Conference on Machine Learning, 2024.
[54] Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127-163, 2000.
[55] Song Mei, Andrea Montanari, and two-layer neural networks. Proceedings of the National Academy of Sciences, 115(33):E7665- E7671, 2018.
Nguyen. mean field view of the landscape of
[56] Yu Meng, Jiaxin Huang, Guangyuan Wang, Chao Zhang, Honglei Zhuang, Lance Kaplan, and Jiawei Han. Spherical text embedding. Advances neural information processing systems, 32, 2019.
[57] Aditya Krishna Menon, Xiaoqian J Jiang, Shankar Vembu, Charles Elkan, and Lucila Ohno- Machado. Predicting accurate probabilities with a ranking loss. In Proceedings of the... International Conference Machine Learning. International Conference on Machine Learn- ing, volume 2012, page 703, 2012.
[58] Péter Mernyei and Cätälina Cangea. Wiki-cs: A wikipedia- benchmark for graph neural networks. arXiv preprint arXiv:2007.02901. 2020.
[59] Alireza Mousavi- Hosseiii, Denny Wu, and Murat A Erdogdu. Learning multi-indes models with neural networks via mean-field langevin dynamics. In The Thirteenth International Conference on Learning Representations, 2025.
[60] Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013.
[61] David F Nettleton. Data mining of social networks represented as graphs. Computer Science Review, 7:1-34, 2013.
[62] Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly- labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLA) pages 188-197, 2019.
[63] OpenAl. GPT text-embedding- https://platfors: openai. cow.docs//d/oceeeeoeooee /own///os//giiiss///// /uides/ embeddings, 2024.
[64] OpenAl. Gpt-4. I-nano model. https://platforn..poe hww.:ppaaattroooooaii....ma openai
1 1-nano, 2025.
[65] Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 36(7):3580-3599 2024.
Opportunities and challenges. Artificial Intelligence Review, 56(1 13:7711--3310, 2023. [66] Ciyuan Peng, Feng Xia, Mchdi Naseriparsa, and Francesco Osborne. Knowledge graphs:
[67] Filippo Radicchi, Santo Fortunato, and Alessandro Vespignani. Citation networks. Models of science dynamics: Encounters between complexity theory and information sciences, pages 233 257, 2011.
[68] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. arXiv preprint (wX.iv1908.100000 2019.
14

---

[69] Bernardino Romera-Paredes. Mohammadamin Barekatain, Alexander Novikov, Matej Balog, Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan Ellenberg, Pengming Wang. Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 625(7995))468 475, 2024.
[70] Marek Rychlik. A proof of convergence of multi-class logistic regression network. arXiv preprini arXiv: /2600, 2019.
[71] Shengyin Sun, Yuxiang Ren, Chen Ma, and Xuecang Zhang. Large language models as topological structure enhancers for text-attributed graphs. arXiv preprint arXiv:2311.13324, 2023.
[72] Jiabin Tang. Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval, pages 491-500, 2024.
[73] Petar Veličković, William Fedus, William Hamilton, Pietro Liò, Yoshua Bengio, and Devon 2018.
Hjelm. Deep graph infomax. arXiv preprint
[74] Vikas Verma, Meng Qu, Kenji Kawaguchi, Alex Lamb, Yoshua Bengio. Juho Kannala, and Jian Tang. Graphmix: Improved training of gnns for semi-supervised learning. In Proceedings of the AAAI conference artificial intelligence, volume 35, pages 10024-1003 2021.
[75] Hanna M Wallach. Topic modeling: beyond bag- bag-of-words. In Proceedings of the 23rd international conference on Machine learning, pages 977-984, 2006.
[76] Duo Wang, Yuan Zuo, Fengzhi Li, and Junjie Wu. Llms as zero-shot graph learners: Alignment of gnn representations with Ilm token embeddings. Advances in Neural Information Processing Systems, 37:5950-555 5973, 2024.
[77] Haoyu Wang, Shikun Liu, Rongzhe Wei, and Pan Li. Model generalization on text attribute graphs: Principles with large language models. arXiv preprint arXiv:2502. /880 2025.
[78] Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang. Community pre- serving network embedding. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
[79] Xinshao Wang, Yang Hua, Elyor Kodirov, Guosheng Hu, Romain Garnier, and Neil M Robertson. Ranked list loss for deep metric learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5207-5216, 2019.
[80] Zehong Wang, Sidney Liu, Zheyuan Zhang, Tianyi Ma. Chuxu Zhang, and Yanfang Ye. Can Ilms convert graphs to text-attributed graphs? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1412-1432, 2025.
[81] Han Xie, Da Zheng, Jun Ma. Houyu Zhang, Vassilis N Ioannidis, Xiang Song, Qing Ping, Sheng Wang, Carl Yang, Yi Xu, et al. Graph-aware language model pre-training on large graph corpus can help multiple graph applications. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5270-5281. 5281, 2023.
[82] Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In International conference on machine learning, pages 5453-5462. PMLR, 2018
[83] Hao Yan. Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al. A comprehensive study on text-attributed graphs: Benchmarking and rethinking. Advances in Neural Information Processing Systems, 36: 36:17238 17264, 2023.
[84] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. technical report, arXiv preprint arXiv:2412.151135 2024.
15

---

[85] Tong Ye. Yangkai Du, Tengfei Ma, Lingfei Wu, Xuhong Zhang, Shouling Ji, and Wenhai Wang. Uncovering 11m- sem--eeertted code: A zero-shot synthetic code detector via code rewriting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 968-976, 2025.
[86] Jianxiang Yu, Yuxiang Ren, Chenghua Gong, Jiaqi Tan, Xiang Li, and Xuecang Zhang Leveraging large language models for node generation in few-shot learning on text-attributed graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 13087-13095, 2025.
[87] Delvin Ce Zhang, Menglin Yang, Rex Ying, and Hady W Lauw. Text-attributed graph representation learning: Methods, applications, and challenges. In Companion Proceedings of the ACM Web Conference 2024, pages 1298-1301, 2024.
[88] Mengmei Zhang, Mingwei Sun, Peng Wang. Shen Fan, Yanhu Mo, Xiaoxiao Xu, Hong Liu, Cheng Yang, and Chuan Shi. Graphtranslator: Aligming graph model to large language model for open-ended tasks. In Proceedings of the ACM Web Conference 2024, pages 1003-1014, 2024.
[89] Zhen Zhang, Meihan Liu, Anhui Wang, Hongyang Chen, Zhao Li, Jiajun Bu, and Bingsheng He. Collaborate to adapt: Source-free graph domain adaptation via bi- di-dircctonna adaptation. In Proceedings of the ACM Web Conference 2024, pages 664 675, 2024.
[90] Zheng Zhang, Yuntong Hu, Bo Pan, Chen Ling, and Liang Zhao. Taga: Text-attributed graph self-supervised learning by synergizing graph and text mutual transformations. arXiv preprint 1rXi:22405.6680 2024.
[91] Huanjing Zhao, Beining Yang. Yukuo Cen, Junyu Ren, Chenhui Zhang, Yuxiao Dong, Evgeny Kharlamov, Shu Zhao, and Jie Tang. Pre-training and prompting for few-shot node classifi- cation on text-attributed graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4467 4478, 2024.
[92] Yusheng Zhao, Junyu Luo, Xiao Luo, Weizhi Zhang, Zhiping Xiao, Wei Ju, Philip S Yu, and Ming Zhang. Multifaceted evaluation of audio-visual capability for milms: Effectiveness, efficiency, generalizability and robustness. arXiv preprint arXiv:2504. 16936, 2025.
[93] Yusheng Zhao, Xiao Luo, Haomin Wen, Zhiping Xiao, Wei Ju, and Ming Zhang. Embrac- ing large language models in traffic flow forecasting. In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, 2025.
[94] Yusheng Zhao, Xiao Luo, Weizhi Zhang, Wei Ju, Zhiping Xiao, Philip S Yu, and Ming Zhang. Marco: Meta-reflection with cross-refeenccing for code reasoning. arXiv preprint arXiv:2505. 17481, 2025.
[95] Yusheng Zhao, Changhu Wang, Xiao Luo, Junyu Luo, Wei Ju. Zhiping Xiao, and Ming Zhang Traci: A data-centric approach for multi- multi-domain generalization on graphs. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pages 13401-13409, 2025.
[96] Yusheng Zhao, Qixin Zhang, Xiao Luo, Junyu Luo, Wei Ju, Zhiping Xiao, and Ming Zhang. Test-time adaptation on graphs via adaptive subgraph- based selection and regularized proto- types. In Forty-second International Conference on Machine Learning. 2025.
[97] Zengqun Zhao, Yu Cao, Shaogang Gong, and Ioannis Patras. Enhancing zero-shot facial expression recognition by I1m knowledge transfer. In 2025 IEEE/CVF Winter Conference Applications of Computer Vision (WACV), pages 815-824. IEEE, 2025.
[98] Yilun Zheng, Sitao Luan, and Lihui Chen. What is missing for graph homophily? disentangling graph homophily for graph neural networks. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[99] Jiong Zhu, Ryan& A Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen K Ahmed, and Danai Koutra, Graph neural networks with heterophily. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11168-11176, 2021.
16

---

[100] Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Be- yond homophily graph neural networks: Current limitations and effective designs. Advances in neural information processing systems, 33:7793-7804, 2020.
[101] Xi Zhu, Haochen Xue, Ziwei Zhao, Wujiang Xu, Jingyuan Huang, Minghao Guo, Qifan Wang, Kaixiong Zhou, and Yongfeng Zhang. Llm as gnn: Graph vocabulary learning for text-attributed graph foundation models. arXiv preprint
2025.
[102] Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. Graphelip: Enhancing transferability in graph foundation models for text-attributed graphs. In Proceedings of the ACM Web Conference 2025, pages 2183-2197, 2025.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope. For details, please refer to Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
In Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

Justification:
In Appendix A, B and C.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: In Section 4.1 and Appendix G.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification: In Appendix and the code url.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips. wttpccccpppccccceeececceccccccct /ttpa//ggudieeeeeeeuuuubiiiiiiii
/CodeSubmic aswinnoolll. onPolicy))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In Section 4.1 and Appendix G.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Repeated experiments over all baselines are costly.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
20

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
In Section 4.1 and Appendix G.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : htt////eeur....eee cc wwtppblcc//cacccccccccccccccccet cew.ehhiiiiiieeeeieieeeeeeeeeeee
Answer: [Yes]
Justification: Ethics.
The research conducted in the paper conforms with the NeurIPS Code of
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: In Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: In Section 4.1 and Appendix G.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
23

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification:
The LLM is used in our algorithm.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https: 220 for what should or should not be described.
24

---

A Proof of Theorem 3.1
Proof. For bundle B, we denote its (predicted) bundle-level class probability distribution as p p(B) (P), .2..... p.)). with corresponding logits as z, Each node in the bundle corresponds to predicted logit vector 2, (Zi,1, 21.2 Zi,C), E B. The bundle-level logits can be written as:
= (21, 22, 22,............................. =c)
2
F12
2₁,
B
B
BEB
SEB
IEB
For an outlier node to in the bundle, it has the (predicted) class probability distribution Po (F1 ,P.)..'''') We also denote their corresponding logits as 2, = (21, ,,, -()). where p. 2oftmax(zo). The most likely class for to is m argmax, [ {p(]i- The bundle label predicted by the LLM ý has its ooe-hot form of y (y1, V2...... ,.))....)). The bundle supervision loss LB and individual supervision loss LIE can be written as:
exp(zi)
c
LB
Σ
yi log
Σ yi log
=
Σ
yi + log L exp(2i):
Σラ=1 exp(zi)
j=1
exp(=1) j== exp(=f)
LIE
Uo log
yi log
B
i=1
1
Vizi log Σ exp(z/)
[
We can then calculate the derivatives as follows. For LB,, we have:
θL
Oz:
a
yi + log
exp(2i)
0z1
dzi
dz
B
i=1
(10)
exp(zi)
V1
(p: Vz)
[B]
i=1 exp(2;) (2))
B
Similarly, we can calculate the derivative of LIE as follows:
dLIE
1
a
Σ
2 + log Σ
exp(())
1
(1) yi).
0=
B
0z
1=1
j=1
For i = m', we have:
1
dLIF
(Pm' ym
(P'm. Vm
0zmm'
Dzym
[B]
Given the condition ú # m and p'm 2 Pm'' we have Fmm = O, and
ÒLBE 1
1
1
1
0LIE a='m
dzh
B
(pm Um
|B]
Pm'
[
P'm
[B]
(p'm. Vmm))
Obviously, Pm 2 O, and therefore,
AL θLIE az'm''
0 S
<
azym
As the probabilities pln is computed from the logits through softmax operations, we have 2f log pl + Const., and therefore we have
θLBE
0LIE alog
5 alog P'mi
25

---

B Proof of Theorem 3.2
Proof. Like Appendix A, for fixed bundle B, denote its (predicted) bundle-level class probability distribution as p(0) _ (B) (pi p2(),,. ..(())), with corresponding logits
More
specifically, we can rewrite the z and p as:
x00) - (2), (0)
1 B
(0):
1
20,
Σ YEB
B
BB
SEB
EEB
exp (x)(=((0)
(0) A
Vi (1,).....C
2==1 _ exp(2:(0))
Moreover, we also denote the one-hot form of bundle label ÿ predicted by the LLM as y (y1, ,2.....,Cc. With this z(0) and y well p(0), then rewrite the cross-entropy loss function (O) as:
C
exp(z:(0)) Σラ= exp(z:(0))
LBE( Σ yo (0) = Σ y: log
--
Evzz (0)+ exp(2:(0))
Σ yizz
after that, according to the chain rule of differentiation, we also can show
(L
(O)
Σ [B] (pi(0) (o,
(11)
(2)
where the final equality follows from (10) in Appendix A. As a result, we have
[V)
(0) yi * (z)l
B
1
(P: - 3al *
(0)loo
B
B
jEB
1
(p(()) yel
V 2g, (0)1l
B
BB
1=1
jEB
2G
(p:(0) + vi)
G
[
where the first equality comes from V (0) ΣjEB V (0), the final inequality comes from
(p) yal S |p:|+ + lysl (0) + ys and the final equality from D Pi( A and Σ V4 A Therefore, we verify the part i) in Theorem 3.2.
Next, we prove the part ii) in Theorem 3.2. At first, show the second-order partial derivatives of the cross-entropy loss function (0) with respect to is also bounded. Similarly, from the result of (L and the chain rule of differentiation, we also can show that
LB) B
( 20) (z
(12)
where the symbol V (P)( denotes the transport of the column gradient (f). Subsequently, from the definition of (0) and the chain rule, we can show that
(p, Vz (0) (0) (0)
(p)
Vz Σ (,(0)
(13)
where the final equality follows from the results that (p:(0)p, when j # i.
0p:(8) (0)
A (0) (1 - p.())) and 8p:(9)
A
(2)
26

---

Merging (13) into (1), we have that
C
pi( (v2(0))))=))
LBE( = (p:(0) 3 (2) +
|B ==1
B
As result, have
m
(p:(0) 3e V2 pi(0) max ([==(0)\V
B
(0) max ([==()))=;(00)'))
G (+E0)10)C3)
(p:(0) - yi * M + PP(
B
1=1
((p.(0) + M + 20 p()) 2(M + G²)
B
where the symbol max (lMI) represents the maximum absolute value among the elements of matrix M and the second inequality follows from max( v²2x < M and max (V=(())V=,(0))7) s
G2 for any j E (1, C..
From the previously established boundedness of the second-order partial derivatives of LB) next show the smoothness of the cross-entropy loss function (B)
Firstly, if we suppose the dimension of the unknown parameter 0 is na, we have that
[V.) - VLBE(62)llz
=
v LEE + + -1- ()) - 02) dAll2
o=
S /nall
LLLEEA00 +1 (1-A)) A)) (0: - 02) ddll
A=
S nd
VDL + +(1 (1- -j)dd
110 - 02112
2,00
where the first equality comes from the fundamental theorem of calculus, the first inequality from ||a|| /m |xc loo where m is the dimension of æ and the final inequality comes from the definition
of (2) [22], i.e., for any matrix A E Ru the (2, -no of matrix is defined as ||AA||..c _ s| Ax/lo x E R", || = 1}.
From the definition of the norm || 112,00 can show that
L'CBE(A0 + -(1 -()d))d
A=
2,00
LB + + -1)]] dA
4na(M + G232
| + (1- 1))D =I
max
|B
where LLLBE (AO, + (1 {))(j :] the j-th line of the Hessian matrix VLL + (1 A)) and
the final incquality follows from the boundedness of the second-order partial derivatives of (B)(
27

---

As result, we have
2nd(M + G2)
[VLBE(()) - VLBE(022]lz _
100 02 2-
[
C Proof of Theorem 3.3
Proof. Note that when the model parameter 0₂ can effectively fit the predicted bundle label g8. namely, 1 E arg (1...( (pe, (B))), we have Lr(0e)) = 0 such that L(O,) LB CR(O,) = LB and (0,,,) = (Dnr(0,) when the iteration index is large. From the results of Theorem 3.2, we know that, when the corresponding first-order and second-order partial derivatives of our adopted GNN 90 are bounded, that is, 2 I G and max( (22,,(00]]) & M where Fi, is the 'c'-th logit of the output vector A (z,,1....,, 2i,0) provided by ge. the cross-entropy loss function LBE( is (M G² -mooohh. Therefore, we have that
LBE(0++1) s (BE + (VCBE(0)) 02) na(M + G2)
||0141 0el1
[
= LBE((,)) - VLBB Ma(M + G2)
110++1 - 0c113
[B]
(14)
= LBE(H)) - -1 V na(M + G²)
?2|(VLBE(())11
1/
[
= LBE((f) - nn- na( na(M M + G
B
where the first inequality follows from the
[2ng(M G²) [B]
smoothness of the adopted cross-entropy
loss function (0) [60] and the three remaining equalities comes from 0.+1 - 0g V Finally, from (14( we have that
ne M + G2)
T
T
n?
(LBE(O))l
Σ (CBE(0.) - LBE(B(+1)) = (0) (0r41).
B
As result, if
we have that,
Ra(M G2j
T
VL (0.)l1
CBE(O1) LBE((T+1) ma(M+ mp)) T
11
18
In other words, limy +cc ET ( 0. Furthermore, from the classic theory of cal-
culus, know that, for positive sequence {aI..... an.... }. if limn Ei 0. then #
an - when n + 00. From this foundational result of calculus and the previous established limitation limy- Σr (C 0, we infer that limy- [VCBe(Or+))ll - such that | LVL(04+))ll V LL (1r+))ll2 + 0, when T + oc.
28

---

D Overall Algorithm
We present the overall algorithm in Algorithm 1.
Algorithm 1 The overall algorithm of our method.
Require: A text-attributed graph g = (V. (yE,T) bundle size nt, and the number of bundles ng-
The set of epochs R where bundle refinement is performed. The total number of epochs for training T.
Ensure: The predicted category of each node in V.
1: for in 1, ..2,.... n.s do
2: Sample
node vzc
3: if' graph G is homophilic graph then
4: Sample (ng nodes from N& (ve) according to Eq.
5:
Construct the bundle using the core node Vi and the (ng - 1) nodes sampled in the previous
step:
6:
else
7:
Sample ng nodes according to Eq. 2 to form the bundle;
8:
end if
9:
Construct the prompt for each bundle using Eq. 3;
10: Query the LLM to obtain the label for the bundle;
end for
12: for i in 2, do
13:
Calculate the entropy-based supervision using Eq. 5;
14:
Calculate the lanking-based supervision using Eq. 7;
15:
Calculate the final loss function using Eq. 8;
16:
Update the parameters in the graph neural network go using gradient descent; if E R then
17:
18:
Update all the bundles using Eq. 9;
19:
end if
20:
end for
21:
Predict the categories of each node in V using the graph neural network Ja-
return The predicted category of each node V.
Table 4: The statistics of the datasets.
Datasets
Cora
CiteSeer
WikiCS
History
Children
Sportsfit
Comell
Texas
Wisc.
Wash.
Nodes
3186
11701
41551
76875
265
Number of Edges
10556
8450
431726
503180
2325044
30201 34
292
310
510
394
Number of Classes
7
10
12
24
13
5
5
Homophily Ratio
0.809
0.764
0.678
0.662
0.464
0,900
0.115
0.067
0.152
0.149
E
Details about the Datasets
We provide more details about the datasets as follows:
Cora. The Cora dataset introduced by McCallum al. [54], consisting of computer science research papers as nodes and their citation links as edges. Each paper is represented by its title and abstract, and the nodes are labeled into 7 classes corresponding to different research topics.
CiteSeer. The CiteSeer dataset originates from the CiteSeer system described by Giles et al. [17]. Similar to Cora, this dataset is also a citation network, with nodes representing papers and edges denoting citation relationships.
WikiCS, The WikiCS dataset is graph of computer science Wikipedia articles proposed by Mernyei and Fazekas [58]. Each node is represented by the text of Wikipedia articles, classified into 10 subfields of computer science.
History, Children, and Sportsfit. These three datasets were constructed by Ni et al. [62] from Amazon co-purchase data. Nodes represent products, edges represent frequent co-purchase relationships, and text attributes contain titles reviews. Each dataset uses a threc-level product
29

---

laxonomy for node labels. The History dataset contains history books as nodes with their co- purchase edges across categories. The Children dataset contains children's 's books nodes connected by their co-purchase relationships as edges across 24 categories. The Sportsfit dataset contains sports and fitness products as nodes with co-purchase edges across 13 categories.
Cornell, Texas, Wisconsin, and Washington. These datasets are collected by Craven al. [11], consisting of web pages from four universities. Nodes are web pages, edges are hyperlinks between pages, and the node attributes are the corresponding page content. Each page is labeled as one of seven types (e.g., student, faculty, department).
We also present the statistics of these datasets in Table 4.
F
Details about the Baseline Methods
We provide more details about the compared baseline methods as follows:
SBERT [68]. Sentence-BERT (SBERT) modifies pretrained BERT with Siamese and triplet networks obtain semantically meaningful sentence representations that be compared with similarity metrics. We use this as a text embedding method.
RoBERTa [49]. RoBERTa is an optimization BERT removing the next-sentence prediction objective, trains longer more data with larger batches, and uses dynamic masking. It achieves better results than BERT by carefully tuning hyperparameters and training recipes.
[63]. OpenAl's (ext-embedding---banee is a text embed- ding model that generates vectorized representations of texts" 2
OpenAT's Sett--mmedding--Laae
LLM2Vee [3]. LLM2Vec an unsupervised solution converting decoder-only LLMs into
powerful text encoders with bidirectional attention, masked next-token prediction, and contrastive learning.
GPT-3.5-turbo [1]. GPT-3.5-turbo is OpenAl's chat-optimized GPT-3.5 model. It offers cost-effective chat model in the GPT-3.5 series at a relatively low cost
GPT-4o/266.. GPT-4o an auto-regressive use this model for fair comparison with Wang et al. [77].
model that is cheaper than GPT-4 4, We
DGI [73]. DGI is an unsupervised approach for learning node embeddings by maximizing mutual information (MI) between local patch embeddings and a global summary of the graph with GCN [33]. We use this as a graph unsupervised learning baseline.
GraphMAE [23]. GraphMAE is masked auto-encoder that reconstructs masked node features using a masking strategy and scaled cosine error loss. We use this as a graph unsupervised learning baseline.
OFA [46]. OFA is a graph foundation model that describes the nodes and edges with natural language, which is then processed by large language models to obtain graph embeddings.
GOFA [34]. GOFA integrates GNN layers into frozen pre-trained LLM to combine semantic and topological modeling abilities. The model is then pretrained on various graph-level tasks.
UniGLM [15]. UniGLM trains a unified graph language model across various attribebi
graphs (TAGs) using a self-supervised contrastive learning objective with positive sampling and a lazy contrastive module.
ZeroG [40]. ZeroG is graph foundation model that encodes node attributes and class semantics via prompts and a prompt-based subgraph sampling module.
GraphGPT [72]. GraphGPT is another graph foundation model that applies graph instruction tuning to LLMs by grounding graph structures in text and uses dual-stage instruction tuning with graph-text alignment.
LLAGA [6]. LLaGA is graph foundation model that translates graph inputs into token embed- dings through structure-aware translation and alignment tuning, preserving graph information in tokens without modifying the base model.
Phups://platfommoopoot wwwp://wwwppppninnommm http://wwwwwweeeeeeeeeeeeeeeenaa
hhupss//platfommm..o
30

---

LLM--PP [77]. LLM-BP is a zero-shot inference method on text-attributed graphs. It proposes adak-ite embeddings and adopts belief propagation with LLM-estimated parameters.
Table
The training time on various datasets.
Datasets
Cora
CiteSeer
WikiCS
History
Children
Sportsfit
Cornell
Texas
Wise.
Wash.
Training Time
11
14
47
59
155
258
20
16
24
20
G Additional Implementation Details
We perform experiments using various datasets. For dataset split, we follow previous works [9, 77] and their official implementations For the text data, also use the data sources from existing works [11, 83, 9, 77] and the version from a Hugging Face repository For the text encoder fe. we use the rask-adaptive embedding proposed in [77]. For the GNN classifier ge. we use GCN [33] for Cora, CiteSeer, WikiCS, History, Children, and Sportsfit. The GCN is implemented with two layers of convolutions with jumping knowledge [82] by default. We use GloGNN [39] for the Cornell, Texas, Wisconsin, and Washington datasets. The hyperparameters of GloGNN follow its official implementation For optimization, we use the Adam optimizer [32] with learning rate 0.001, and optimize for 500 epochs by default. For bundles, the bundle size is set to 5, and the number of bundles is set to 100. For hardware, we use an NVIDIA RTX 3090 GPU with 24GB memory. As the time for querying GPTs from online sources depends on network conditions, we only measure time for training GNNs Table
Table 6: The prompt parameters of different datasets.
Dulasets
Caode, description>
sgraph, description>
<aode_ zefer nane ploral>>>>
Soodo refer nano. ringular>
Cura
opening
papers
learning
cilaton network
papers
Paper
Ciesseer
description opening test of
seientiic
cination netwoek
papers
WikiCS
entry and content of wikipodia
knowledge graph
entries
Entry
History
deserption litle of the book
e-commerce network
books
Book
description or
child
Children
literature
e-commerce network
books
Sporishi the lifl of good sports & litness e-commerce network
products
Product
webpage
university webpage
webpages
Webpuge
Texas
webyuge text
university wchpige network
webjuges
Wehpuge
Wisconsin
webpage
universiry webpage nerwork
wehpages
Wehpage
Washingt webysgge text
university wchpage network
webpuges
Webpuge
H Prompt Details
We also provide additional details about the prompt. Specifically, given the prompt template in Figure 4, we fill the prompt parameters using the texts in Table 6 together with the number of classes in Table and specific classes attached to each dataset to form the final prompts.
httpps///gtuubbbmo///gggh//////t BP
htnps//gghibbbooo/cccccessnnnnnn
31


Reviews:
[{"review_id": "EER7p389WZ", "reviewer": "Reviewer_E32F", "summary": "This paper addresses the challenge of zero-shot inference on text-attributed graphs (TAGs), where the goal is to classify nodes without any labeled data. The paper further introduces a new method called Dynamic Text Bundling Supervision (DENSE). Instead of querying an LLM with individual node texts, DENSE groups nodes with close proximity (either topological or semantic) into \"bundles.\" The LLM is then queried with the combined text of these bundles to determine a single \"bundle label\", which represents the most frequent category within that bundle. These bundle-level labels are then used to supervise the training of a graph neural network (GNN). The authors provide theoretical justification for their method, showing its tolerance to outliers and analyzing its convergence properties. The effectiveness of DENSE is demonstrated through extensive experiments on ten datasets, where it consistently outperforms fifteen baseline methods, including various text encoders, generative LLMs, and graph learning models.", "questions": "See the weaknesses above.", "limitations": "See the weaknesses above.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n1. The problem is novel. The paper studies a relatively new problem of zero-shot inference on text-attributed graphs using LLMs. Although there are some recent efforts using LLMs on graphs, applying them for zero-shot node classification without any labels is a challenging problem. The paper correctly identifies the key bottlenecks of information limitation and response unreliability, making the problem well-defined and highly relevant.\n\n2. The method is quite innovative. The core idea of \"dynamic text bundling\" is a novel and intuitive solution to the identified problem. By bundling texts, the method provides richer, contextualized information to the LLM, moving beyond isolated node attributes. The supervision and refinement mechanisms are well-reasoned; the bundle-level supervision is theoretically shown to be more tolerant to noisy labels than individual supervision, and the dynamic refinement process actively purges noise. This multi-faceted approach is a clear methodological innovation.\n\n3. The experiment is extensive. The authors test their method across ten diverse datasets, covering both homophilic and heterophilic graphs, as well as various domains like citation, e-commerce, and webpage networks. They compare DENSE against a wide spectrum of 15 recent and relevant baselines, including text encoders, generative LLMs, graph self-supervised methods, and dedicated graph foundation models. The consistent outperformance across all datasets strongly validates the method's effectiveness and generalizability. The inclusion of ablation studies and analysis of different LLM backbones further strengthens the empirical claims.\n\n### Weaknesses\n\n1. The exploration of heterophily is limited. While the paper includes heterophilic graphs (Cornell, Texas, Wisconsin, Washington), the mechanism for handling them relies solely on switching from \"topological proximity\" to \"semantic proximity\" for bundle sampling. This semantic sampling is based on the initial text embeddings. The paper could benefit from a deeper exploration of how the bundling strategy interacts with strong heterophily, where even semantically similar nodes might belong to different classes. The performance gains on heterophilic graphs, while strong, might stem more from the power of the base GNN designed for heterophily (GloGNN) rather than the bundling strategy itself being optimally adapted for this challenging scenario.\n\n2. The method depends on High-Quality Initial Embeddings: The \"semantic proximity\" sampling, crucial for heterophilic graphs, relies on the quality of initial node embeddings generated by a text encoder. The paper uses a task-adaptive embedding method from a prior work. If these initial embeddings are poor or do not capture the relevant semantic nuances for classification, the resulting bundles could be noisy from the outset, potentially hampering the entire process. The paper does not analyze the sensitivity of the method to the quality of these initial text representations.\n\n3. The experiment lacks sufficient large-scale datasets for evaluation. Most datasets used for evaluation are relatively small. Four of the ten datasets (Cornell, Texas, Wisconsin, Washington) have fewer than 300 nodes each, and two others (Cora, CiteSeer) are also considered small by modern standards. While the inclusion of datasets like Sportsfit (173k nodes) is commendable, the method's scalability and effectiveness on truly large-scale graphs with millions of nodes remain under-explored. The default setting uses 100 bundles for supervision, which may provide very sparse coverage on a much larger graph, potentially requiring a significant increase in costly LLM queries to achieve good results, as evidenced by the finding that more bundles yield better performance.", "was_revised": true, "final_justification": "This paper addresses the challenge of zero-shot inference on text-attributed graphs (TAGs), where the goal is to classify nodes without any labeled data. Overall, the problem is interesting and the proposed method is novel.\n\nAt the beginning, I had some concerns of the method and experiments, but during the rebuttal phase, the authors successfully addressed my concerns. Therefore, I will give an accept.", "rebuttal": "We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.\n\n> Q1. The exploration of heterophily is limited. While the paper includes heterophilic graphs (Cornell, Texas, Wisconsin, Washington), the mechanism for handling them relies solely on switching from \"topological proximity\" to \"semantic proximity\" for bundle sampling. This semantic sampling is based on the initial text embeddings. The paper could benefit from a deeper exploration of how the bundling strategy interacts with strong heterophily, where even semantically similar nodes might belong to different classes. The performance gains on heterophilic graphs, while strong, might stem more from the power of the base GNN designed for heterophily (GloGNN) rather than the bundling strategy itself being optimally adapted for this challenging scenario.\n\n**A1.** Thank you for the comment. While some semantically similar nodes might belong to different classes, they are generally close in the label space. Additionally, the proposed bundle supervision is tolerant to outliers (Theorem 3.1). As for the performance gain, we compare the accuracy of individual query and bundle query below. The results suggest that the bundling strategy using semantic proximity is effective.\n\n|Datasets|Texas|Cornell|Wisconsin|Washington|\n|-|-|-|-|-|\n|Individual Query|84.49|71.73|75.84|72.05|\n|Bundle Query|92.51|84.82|87.17|81.66|\n\n\n> Q2. The method depends on High-Quality Initial Embeddings: The \"semantic proximity\" sampling, crucial for heterophilic graphs, relies on the quality of initial node embeddings generated by a text encoder. The paper uses a task-adaptive embedding method from a prior work. If these initial embeddings are poor or do not capture the relevant semantic nuances for classification, the resulting bundles could be noisy from the outset, potentially hampering the entire process. The paper does not analyze the sensitivity of the method to the quality of these initial text representations.\n\n**A2.** Thank you for the comment. We show that our method is robust to the quality of initial text embeddings. Specifically, for embedding $\\textbf x$, we introduce noisy using $\\textbf x \\leftarrow (1 + \\beta \\textbf n)\\odot \\textbf x$, where $\\textbf n\\sim\\mathcal N(\\textbf 0,\\textbf I)$ is gaussian noise. We show the performance of our method under different noise ratios $\\beta$ on the WikiCS dataset below. As can be seen from the results, our method is robust when the quality of text embeddings degrades. \n\n|Noise Ratio|0|0.05|0.1|0.15|\n|-|-|-|-|-|\n|DENSE|71.03|70.94|70.82|70.50|\n\n> Q3. The experiment lacks sufficient large-scale datasets for evaluation. Most datasets used for evaluation are relatively small. Four of the ten datasets (Cornell, Texas, Wisconsin, Washington) have fewer than 300 nodes each, and two others (Cora, CiteSeer) are also considered small by modern standards. While the inclusion of datasets like Sportsfit (173k nodes) is commendable, the method's scalability and effectiveness on truly large-scale graphs with millions of nodes remain under-explored. The default setting uses 100 bundles for supervision, which may provide very sparse coverage on a much larger graph, potentially requiring a significant increase in costly LLM queries to achieve good results, as evidenced by the finding that more bundles yield better performance.\n\n**A3.** Thank you for your comment. We provide results on ogbn-products with 2,449,029 nodes and 61,859,140 edges. The baseline LLMBP uses 940 queries (default setup), while the proposed DENSE uses 100 and 200 queries. The results suggest that DENSE outperforms the baseline with only 100 bundles (default setup, much less than the baseline), and the performance further improves with additional bundles.\n\n|Methods|LLMBP|DENSE (100 bundles)|DENSE (200 bundles)|\n|-|-|-|-|\n|ogbn-products|74.71|76.88|79.91|\n\nIn light of these responses, we hope that we have addressed your concerns and that you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points."}, {"review_id": "9HkbdiZ8KV", "reviewer": "Reviewer_zC3m", "summary": "This paper proposes dynamic text bundling supervision for zero-shot inference on text-attributed graphs. The authors propose to query the LLMs with text bundles to improve the robustness of label and subsequent supervision. The bundles are further refined to reduce noise and improve the performance. Additionally, the authors provide theoretical analysis of their method. Comprehensive experiments are also provided and the proposed method achieves state-of-the-art performance on many datasets.", "questions": "Please refer to the weakness section.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The proposed text bundling is novel and interesting. The idea of bundling text pieces to query LLMs is different from many existing methods. The writing of this paper is clear and easy to understand.\n2. The proposed method is supported by solid mathematical analysis, with theorems, detailed proofs and explanations of their implications. \n3. The authors perform extensive experiments on a wide range of graph benchmark datasets, and the proposed method outperforms many state-of-the-art baselines.\n\nWeaknesses:\n1. The authors adopt two bundle sampling techniques: topological proximity and semantic proximity, according to graph homophily. In practice, it may be non-trivial to determine the homophily degree of graphs, and this paper lacks discussion on this. The authors should discuss how to decide graph homophily when it is not obvious from the graph's meta information.\n2. The motivation of the proposed ranking-based supervision is not very clear. Why do you use both entropy-based and ranking-based supervision? Why using a min operator? The authors should explain the rationale behind this technique.\n3. The authors only use one GNN architecture for homophilic graphs, while there are many GNN architectures available. The authors should perform experiments on different types of GNNs like GraphSAGE or GIN to show the influence of GNN architectures.", "was_revised": true, "final_justification": "My concerns have been addressed, thus I vote for acceptance.", "rebuttal": "We are truly grateful for the time you have taken to review our paper, your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.\n\n> Q1. The authors adopt two bundle sampling techniques: topological proximity and semantic proximity, according to graph homophily. In practice, it may be non-trivial to determine the homophily degree of graphs, and this paper lacks discussion on this. The authors should discuss how to decide graph homophily when it is not obvious from the graph's meta information.\n\n**A1.** Thank you for your suggestion! We will discuss the graph homophily in the revised version. Specifically, one approach is to sample a set of connected node pairs and directly ask the LLM whether they belong to the same category. Below are the results for ground truth homophily and the prediction by GPT4o-mini.\n\n|Datasets|Cora|Citeseer|History|Children|Sportsfit|Wikics|Cornell|Texas|Wisconsin|Washington|\n|---|---|---|---|---|---|---|---|---|---|---|\n|GT|0.81|0.76|0.66|0.46|0.90|0.67|0.11|0.06|0.15|0.19|\n|Predicted|0.70|0.81|0.73|0.35|0.81|0.52|0.05|0.04|0.06|0.02|\n\n\n> Q2. The motivation of the proposed ranking-based supervision is not very clear. Why do you use both entropy-based and ranking-based supervision? Why using a min operator? The authors should explain the rationale behind this technique.\n\n**A2.** Thank you for your question! We use entropy-based supervision because nodes in the bundle are more likely to fall into the mode category of the bundle. Since there may be nodes from other categories, the ranking-based supervision is designed to penalize bundles in which the majority of nodes are not classified into the mode category. The min operator is used to achieve this functionality. We will provide more explanation of the rationale in the revised version.\n\n> Q3. The authors only use one GNN architecture for homophilic graphs, while there are many GNN architectures available. The authors should perform experiments on different types of GNNs like GraphSAGE or GIN to show the influence of GNN architectures.\n\n**A3.** Thank you for your suggestion! We provide additional results using GraphSAGE as follows. The results suggest that our method achieves good performance with different GNNs.\n|Datasets|Cora|CiteSeer|History|\n|-|-|-|-|\n|DENSE w/ GCN|75.09|72.37|67.31|\n|DENSE w/ GraphSAGE|76.38|74.86|66.97|\n\nWe will properly incorporate your suggestions into our revised version. Thanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions."}, {"review_id": "aJ9IBVniqp", "reviewer": "Reviewer_4o6p", "summary": "This paper proposes DENSE (Dynamic Text Bundling Supervision), a novel framework for zero-shot inference on text-attributed graphs (TAGs). The method addresses two key challenges: (1) LLMs' limited access to graph structural information and (2) unreliability of LLM responses due to hallucination or insufficient context. DENSE samples node bundles (based on topological/semantic proximity), queries LLMs for bundle-level labels, uses these labels to supervise GNN training, and dynamically refines bundles to exclude noisy nodes. Theoretical analysis demonstrates tolerance to outliers and convergence guarantees. Extensive experiments on 10 diverse TAG datasets show state-of-the-art performance against 15 baselines.", "questions": "See Weaknesses.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. Novel & impactful framework: DENSE innovatively bridges LLMs and GNNs via dynamic text bundling, solving LLMs' structural unawareness and unreliability for zero-shot TAG inference. The bundle-supervise-refine pipeline is elegant and practical.\n\n2. Strong empirical validation: Outperforms 15 baselines across 10 diverse datasets (homophilic/heterophilic). Adaptive sampling (topological/semantic) and ablation studies prove method versatility.\n\n3. Theoretical grounding: Formal proofs show bundle supervision’s outlier tolerance and convergence guarantees, significantly boosting credibility beyond empirical results.\n\n4. High reproducibility: Public code, detailed prompts, and exhaustive training specs set a commendable standard for transparency.\n\nWeaknesses\n\n1. Scope expansion: Method currently requires textual attributes; exploring LLM-generated pseudo-descriptors for non-text-attributed graphs could broaden applicability.\n\n2. Heterophily refinement: While semantic sampling handles heterophily, explicitly prompting LLMs to resolve topological proximity vs. semantic conflicts (e.g., \"identify dominant label in semantically divergent bundles\") may further boost robustness.\n\nDENSE‘s innovations address critical LLM-GNN integration gaps. Weaknesses reflect optional extensions, not flaws.", "was_revised": true, "final_justification": "Thank you for the response, and I will keep my positive score.", "rebuttal": "We are truly grateful for the time you have taken to review our paper, your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.\n\n> Q1. Scope expansion: Method currently requires textual attributes; exploring LLM-generated pseudo-descriptors for non-text-attributed graphs could broaden applicability.\n\n**A1.** Thank you for your suggestion! We will add the following discussion to the revised manuscript: The proposed DENSE framework can be extended to a more general setting where text attributes are not provided. Non-text-attributed graphs can be converted to text-attributed graphs, which have been explored by Wang et al. [1]. We will introduce these techniques to expand the scope of our method in future work.\n\n[1] Can LLMs Convert Graphs to Text-Attributed Graphs? Wang et al. 2024\n\n> Q2. Heterophily refinement: While semantic sampling handles heterophily, explicitly prompting LLMs to resolve topological proximity vs. semantic conflicts (e.g., \"identify dominant label in semantically divergent bundles\") may further boost robustness.\n\n**A2.** Thank you for your suggestion! We will extend our work by exploring the prompting techniques in bundle query when handling heterophilic graphs, explicitly informing the LLMs about the homophilic/heterophilic information of the current graph to improve the robustness of our method.\n\nWe will properly incorporate your suggestions into our revised version. Thanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions."}, {"review_id": "Z2QLBXHla8", "reviewer": "Reviewer_engv", "summary": "This paper addresses zero-shot inference on text-attributed graphs (TAGs) using large language models (LLMs). The authors identify two key challenges: (1) LLMs receive limited information about graph structure, and (2) LLMs produce unreliable responses due to hallucination and information insufficiency. To address these issues, they propose Dynamic Text Bundling Supervision (DENSE), which queries LLMs with bundles of topologically or semantically similar texts rather than individual texts. The method obtains bundle-level labels from LLMs and uses these to supervise graph neural networks through specially designed entropy-based and ranking-based loss functions. The approach includes a bundle refinement mechanism that removes noisy nodes during training. Theoretical analysis is provided to justify the design choices, and extensive experiments on ten datasets demonstrate the effectiveness of the proposed method.", "questions": "1. Regarding weaknesses 4, could you clarify if the experimental settings are consistent across all comparisons? What specific configurations were used for baseline methods?\n\n2. The method proposed in this paper is a kind of LLMs for Data Augmentation, but the baselines do not include similar experimental settings.（such as Chen, Z., Mao, H., Wen, H., Han, H., Jin, W., Zhang, H., Liu, H., and Tang, J. Label-free node classification on graphs with large language models (llms). ICLR, 2024e.）\n\n3. Why is the bundle refinement limited to only two fixed epochs (300 and 400) with removal of just the least confident node? Have you explored more adaptive refinement strategies or different refinement frequencies? What is the sensitivity to these hyperparameters?\n\n4. For heterophilic graphs, you use semantic proximity for sampling, but this essentially ignores the graph topology entirely. How does this approach truly address the claimed limitation of \"limited information on graph structure\"? Could you provide more analysis on how graph structure is actually utilized?\n\n5. The method requires multiple LLM queries (100 bundles × datasets). Could you provide analysis of the computational cost and scalability compared to baselines?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper provides solid theoretical foundations with three theorems proving the tolerance to outliers and convergence properties of the proposed bundle supervision. The experimental evaluation is comprehensive, covering ten datasets across different domains and graph types (homophilic and heterophilic);\n\n2. The work addresses an important problem in graph learning and provides a  solution that outperforms baselines across various datasets. The bundle-based approach offers a new perspective on leveraging LLMs for graph tasks;\n\n3. The paper is generally well-written with clear motivation, methodology description, and experimental setup. The figures effectively illustrate the key concepts, particularly Figure 1 showing the difference between individual and bundle queries;\n\n4. The method shows improvements across different types of graphs and is agnostic to specific GNN architectures, making it broadly applicable.\n\nWeaknesses:\n1. The core idea of bundling similar items for better LLM performance is not particularly novel. The contribution is more incremental, combining existing techniques (LLM querying + GNN training) rather than introducing fundamentally new concepts;\n\n2. The approach treats LLMs and GNNs as separate components rather than achieving true integration. The LLM serves merely as a labeling tool, and there's limited synergy between the two components;\n\n3. The bundle refinement process is quite mechanical, removing only the least confident nodes at fixed epochs (300 and 400), which may not be optimal for all scenarios.\n\n4. Although the experimental results were comprehensive and effective, I still have doubts about the reliability of the experimental results (as shown in table 1 and table 3). Table3 v2 and v5 correspond to single-node annotation and individual supervision respectively. Their results are also superior to most baselines, which seems to be contrary to the core contribution of this paper.", "was_revised": true, "final_justification": "I have reviewed the authors' responses, and they have addressed most of my concerns.", "rebuttal": "We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.\n\n> Q1. The core idea of bundling similar items for better LLM performance is not particularly novel. The contribution is more incremental, combining existing techniques (LLM querying + GNN training) rather than introducing fundamentally new concepts;\n\n**A1.** Thank you for the comment. The contribution of this paper is three-fold:\n* **The idea of bundling nodes/texts**: We propose to bundle relevant nodes and text attributes, and use these bundles instead of individual nodes to query LLMs and supervise GNNs.\n* **Unified Framework**: We propose a novel framework consisting of bundle sampling, bundle query, bundle supervision, and bundle refinement. The proposed framework outperforms a number of baselines.\n* **Theoretical Analysis**: We provide rigorous theoretical analysis of our method, showing its tolerance to outlier nodes and the convergence properties.\n\nOur method is not a simple combination of existing techniques. We introduce the new concept of bundles, while prior studies adopt individual supervision on GNNs.\n\n> Q2. The approach treats LLMs and GNNs as separate components rather than achieving true integration. The LLM serves merely as a labeling tool, and there's limited synergy between the two components;\n\n**A2.** Thank you for the comment. This paper explores graph annotation from the perspective of bundles, which are used to facilitate downstream GNN training.\n\n> Q3. The bundle refinement process is quite mechanical, removing only the least confident nodes at fixed epochs (300 and 400), which may not be optimal for all scenarios.\n\n**A3.** Thank you for the comment. As the bundle size of our paper is 5, removing one item is a reasonable choice. We also provide the results of removing more items below. For the epochs, we also provide results of alternative configurations. The results suggest the current default configuration achieves satisfactory performance.\n\n|Dataset|Citeseer|History|\n|-|-|-|\n|Default|72.37|67.31|\n|Removing 2 items each time|72.06|67.12|\n|Removing at 200, 300, and 400 epochs|71.94|67.21|\n\n\n> Q4. Although the experimental results were comprehensive and effective, I still have doubts about the reliability of the experimental results (as shown in table 1 and table 3). Table3 v2 and v5 correspond to single-node annotation and individual supervision respectively. Their results are also superior to most baselines, which seems to be contrary to the core contribution of this paper.\n\n**A4.** Thank you for the comment. For V2, our method achieves significant improvement compared to individual querying. For V5, while it uses $\\mathcal L_{IE}$ (defined in Theorem 3.1), it still adopts the overall framework of DENSE, including bundle sampling/query, ranking-based supervision, and bundle refinement. Many baseline methods suffer from poor generalization capability, leading to low accuracy under the zero-shot setting. For example, GOFA and ZeroG adopt joint training of LLM and graphs. Their performance worsens when the test distribution is different from the training distribution.\n\n> Q5. Regarding weaknesses 4, could you clarify if the experimental settings are consistent across all comparisons? What specific configurations were used for baseline methods?\n\n\n**A5.** Thank you for the question. The experimental setting is zero-shot inference on TAGs without access to node labels, and it is consistent across all comparisons. We provide the details as follows:\n* Text encoders (SBERT, RoBERTa, TE-3-Large, LLM2Vec): The text attributes and class labels are encoded by the text encoders, and the nodes are assigned according to the cosine similarity between their text embeddings and class embeddings.\n* Generative LLMs (GPT-3.5-Turbo, GPT-4o): The class labels are obtained by directly prompting the LLMs.\n* Graph SSL methods (DGI, GraphMAE): The implementation follows the TSGFM [1] benchmark, and the models are trained on the ogbn-arxiv dataset.\n* OFA, ZeroG, GraphGPT, LLAGA are trained on the ogbn-arxiv dataset.\n* For GOFA, the pretrained weights are adopted, and the ogbn-arxiv dataset is used for finetuning.\n* For UniGLM, the pretrained model is directly adopted for inference.\n* For LLMBP, it is a pretraining-free solution, and we directly use their official implementation.\n\n[1] Text-space graph foundation models: Comprehensive benchmarks and new insights. Chen et al. NeurIPS 2024.\n\n> Q6. The method proposed in this paper is a kind of LLMs for Data Augmentation, but the baselines do not include similar experimental settings.（such as Chen, Z., Mao, H., Wen, H., Han, H., Jin, W., Zhang, H., Liu, H., and Tang, J. Label-free node classification on graphs with large language models (llms). ICLR, 2024e.）\n\n**A6.** Thank you for bringing this excellent work to our attention! We will include this work in the revised version. We want to clarify that our method is not data augmentation. We provide a comparison between our method and Chen et al. as follows:\n|Datasets|CiteSeer|WikiCS|ogbn-products|\n|-|-|-|-|\n|DENSE (ours)| 72.37 | 71.03 | 76.88 |\n|Chen et al. | 68.64 | 64.72 | 74.54 |\n\n> Q7. Why is the bundle refinement limited to only two fixed epochs (300 and 400) with removal of just the least confident node? Have you explored more adaptive refinement strategies or different refinement frequencies? What is the sensitivity to these hyperparameters?\n\n**A7.** Thank you for the question. For the time of refinement, the general principle is to perform refinement after the training converges. In practice, we observe that the bundle supervision converges within 300 epochs, and that after refinement, it converges within an additional 100 epochs. Additionally, since the optimal bundle size is 5 (Section 4.4), removing one item each time is reasonable. We also provide results of different configurations in A3. The model's performance is not very sensitive to the configuration.\n\n> Q8. For heterophilic graphs, you use semantic proximity for sampling, but this essentially ignores the graph topology entirely. How does this approach truly address the claimed limitation of \"limited information on graph structure\"? Could you provide more analysis on how graph structure is actually utilized?\n\n**A8.** Thank you for the question. For the homophilic graphs, graph topology is important for understanding the semantics of nodes, and we incorporate the topological information in bundle sampling. For heterophilic graphs, the topology provides less information, and we find semantic proximity more effective in bundle sampling.\n\n> Q9. The method requires multiple LLM queries (100 bundles × datasets). Could you provide analysis of the computational cost and scalability compared to baselines?\n\n**A9**. Thank you for the question. We provide the computational cost and scalability on the Cora dataset below. The results show that our method is better than the baseline. The computational cost is generally not a concern, as GNNs are lightweight models and our method maintains good performance even with cheaper LLMs (e.g. DeepSeek-V3 or Gemini-2.5-flash in Table 2).\n\n|Methods|Number of LLM queries|Total Inference Time (s)|\n|-|-|-|\n|LLMBP|140|34|\n|DENSE (ours)|100|32|\n\nIn light of these responses, we hope that we have addressed your concerns and that you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points."}, {"review_id": "AyTtgAy2VO", "reviewer": "Reviewer_F3wU", "summary": "To apply Large language models in text attributed graphs has drawn increasing attention, this paper proposes Dynamic Text Bundling Supervision that queries Large language models with bundles of texts. Bundle-level labels are obtained and used in supervised tasks for graph neural networks. The proposed approach mainly addresses two challenges: Large language models only receive limited information on graph structure and unreliable responses from Large language models. Theoretical analysis of the proposed method is provided. Experiments across ten datasets are conducted and show improved performance over existing methods.", "questions": "1. Can a text bundle be interpreted as context window size? Any insight? If a bundle is interpreted as a proxy for this window—composed of graph nodes and their descriptions—this has important implications for scaling, quality of encoding, and model performance.\n\n2. Any other way to tell graphs with high homophily or heterophily other than graph type? What to do with medium homophily and heterophily? The design of the bundle sampling mechanism may need to be adaptive, possibly using homophily-aware selection or weighting strategies. \n\n3.The paper states: “During optimization of the graph neural network gθ, the bundle B may include nodes that do not belong to the category of yˆB.”\n\nGiven that this is a supervised learning setup and labels are available during training, it’s unclear why the bundles are not explicitly constructed to include only same-class nodes. Such a strategy could improve signal consistency in the text prompts and reduce label noise within the bundle. If there are limitations that prevent such a strategy—such as low label density or design constraints—these need to be explicitly stated. Otherwise, this decision appears to introduce noise and reduces the clarity of the supervision signal.\n\n4. \"Bundle\" seems to be the central component of the proposed method:\n\n(1) Edges in a bundle is implicitly input to Large language models as graph structure information\n\n(2) Multiple  nodes in a bundle provides a large context, which will reduce hallucination\n\nHowever, more nodes are preferred in a bundle to incorporate more structural information, which also increases \"outliers\". As the core component, analysis in 4.4 is not sufficient.\n\nStructural information incorporated in this way is quite limited.\n\n5. It is better to include dataset split details directly in the paper instead of referring to other papers.\n\n6. Performance shown in experiments often is far below traditional GNN methods (e.g., GCN achieves 81.5% accuracy on Cora, GraphSage achieves 83.67% on Wiki CS). As the proposed LLMs-based approach does not modify LLMs in any way, simple prompt-level work may have limitations. Please justify the motivation using LLMs, just because they are popular?", "limitations": "Adequate.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n+ Large language models may serve as an effective knowledge base for supervised tasks in graph neural networks.\n\n+ Theoretical analysis of the proposed method is provided. \n\n+ Experiments across ten datasets are conducted and show improved performance over other LLM-based methods.\n\nWeaknesses\n\n1. The significance of this paper is limited. There are some technical concerns. Please refer to questions for details.\n\n2. The theoretical analysis focuses on bundle, which is quite limited as supervised tasks are mainly performed by Large language models.", "was_revised": true, "final_justification": "The responses from authors addressed most of my questions. I raised the rating accordingly.", "rebuttal": "We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.\n\n> Q1. The significance of this paper is limited. There are some technical concerns. Please refer to questions for details.\n\n**A1.** Thank you for the comment. The contribution of this paper is three-fold:\n* **The idea of bundling nodes/texts**: We propose to bundle relevant nodes and text attributes, and use these bundles instead of individual nodes to query LLMs and supervise GNNs.\n* **Unified Framework**: We propose a novel framework consisting of bundle sampling, bundle query, bundle supervision, and bundle refinement. The proposed framework outperforms a number of baselines.\n* **Theoretical Analysis**: We provide rigorous theoretical analysis of our method, showing its tolerance to outlier nodes and the convergence properties.\n\nWe will address your technical concerns in the following.\n\n> Q2. The theoretical analysis focuses on bundle, which is quite limited as supervised tasks are mainly performed by Large language models.\n\n**A2.** Thank you for the comment. The bundle is an important concept of this paper. Bundles are used for supervising the GNN, which is then used for the downstream task. The two theorems in this paper focus on bundle supervision, which is one of the core components of this paper.\n\n> Q3. Can a text bundle be interpreted as context window size? Any insight? If a bundle is interpreted as a proxy for this window—composed of graph nodes and their descriptions—this has important implications for scaling, quality of encoding, and model performance.\n\n**A3.** Thank you for the question. A text bundle is not the context window size. In addition to providing extra information, text bundling has the following advantages. Firstly, text bundling reduces the task difficulty for LLMs, as predicting the overall property of a set of closely related text items is easier than predicting one of them. This makes the supervision signals more robust. Secondly, text bundling is more efficient in supervising GNNs, as all nodes in a bundle can provide supervision signals.\n\n> Q4. Any other way to tell graphs with high homophily or heterophily other than graph type? What to do with medium homophily and heterophily? The design of the bundle sampling mechanism may need to be adaptive, possibly using homophily-aware selection or weighting strategies.\n\n**A4.** Thank you for the question. The homophilic ratio can be estimated by sampling a set of connected node pairs and directly asking the LLM whether they belong to the same category. Below are the results for ground truth homophily and the prediction by GPT4o-mini. The predicted results are close to the ground truth, which is enough to roughly decide the type of bundle sampling method to use.\n\n|Datasets|Cora|Citeseer|History|Children|Sportsfit|Wikics|Cornell|Texas|Wisconsin|Washington|\n|---|---|---|---|---|---|---|---|---|---|---|\n|GT|0.81|0.76|0.66|0.46|0.90|0.67|0.11|0.06|0.15|0.19|\n|Predicted|0.70|0.81|0.73|0.35|0.81|0.52|0.05|0.04|0.06|0.02|\n\nWe also provide an adaptive approach of bundle sampling, where we take the union of topological sampling and semantic sampling, with the ratio of each determined by the predicted homophily. The results are shown below, where we can see that adaptive sampling also outperforms the baseline.\n\n|Datasets|Citeseer|Children|Cornell|\n|-|-|-|-|\n|LLMBP|69.51|24.81|83.28|\n|DENSE, adaptive sampling|72.14|30.62|84.82|\n\n> Q5. The paper states: “During optimization of the graph neural network gθ, the bundle B may include nodes that do not belong to the category of yˆB.”\n> Given that this is a supervised learning setup and labels are available during training, it’s unclear why the bundles are not explicitly constructed to include only same-class nodes. Such a strategy could improve signal consistency in the text prompts and reduce label noise within the bundle. If there are limitations that prevent such a strategy—such as low label density or design constraints—these need to be explicitly stated. Otherwise, this decision appears to introduce noise and reduces the clarity of the supervision signal.\n\n**A5.** Thank you for the comment. We want to clarify that the setting of this paper is zero-shot inference instead of supervised learning. In zero-shot inference, **we do not have access to the ground truth labels**.\n\n> Q6. \"Bundle\" seems to be the central component of the proposed method:\n> (1) Edges in a bundle is implicitly input to Large language models as graph structure information\n> (2) Multiple nodes in a bundle provides a large context, which will reduce hallucination\n> However, more nodes are preferred in a bundle to incorporate more structural information, which also increases \"outliers\". As the core component, analysis in 4.4 is not sufficient.\n> Structural information incorporated in this way is quite limited.\n\n**A6.** Thank you for the comment. There is a balance between incorporating more information and excluding outliers. We empirically find that the bundle size of 5 is optimal in terms of overall accuracy. Additionally, by comparing bundle query and individual query in Section 4.3, we show that bundling improves performance. We also provide an additional experiment where we manually include one outlier into the bundles. The results are shown below, and we can see that our method is robust to outliers, which is also demonstrated by Theorem 3.1.\n\n|Datasets|Cora|History|Texas|\n|-|-|-|-|\n|Individual query|71.96|63.95|84.49|\n|Bundle query|75.09|67.31|92.51|\n|Bundle query with a manually included outlier|74.72|66.59|91.44|\n\n> Q7. It is better to include dataset split details directly in the paper instead of referring to other papers.\n\n**A7.** Thank you for your suggestion. As our setting is zero-shot inference, there is no training set, and we list the test set split details as follows.\n\n|Datasets|Cora|Citeseer|Wikics|History|Children|Sportsfit|Cornell|Texas|Wisconsin|Washington|\n|---|---|---|---|---|---|---|---|---|---|---|\n|Test Split|542|2566|5847|8311|15375|121139|191|187|265|229|\n\n\n> Q8. Performance shown in experiments often is far below traditional GNN methods (e.g., GCN achieves 81.5% accuracy on Cora, GraphSage achieves 83.67% on Wiki CS). As the proposed LLMs-based approach does not modify LLMs in any way, simple prompt-level work may have limitations. Please justify the motivation using LLMs, just because they are popular?\n\n**A8.** Thank you for your question. **The setting of this paper is zero-shot inference**, while the performance you mentioned (81.5% and 83.67%) is semi-supervised learning. LLMs have strong zero-shot generalization ability, and they can understand the semantics of text attributes and class labels, enabling preliminary classification, which is beyond the capabilities of traditional GNNs.\n\nIn light of these responses, we hope that we have addressed your concerns and that you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points."}]

Author rebuttal:

Meta-review: The paper is in the space of using LLMs for graphs. In particular, the paper investigates zero-shot node classification on text-attributed graphs (TAGs) Instead of querying LLMs with isolated node texts, the authors propose Dynamic Text Bundling Supervision (DENSE) which forms bundles of semantically or topologically proximate nodes, queries the LLM for a bundle-level label, and uses these labels to supervise a GNN plus iterative bundle refinement. It is argued that this bundling approach helps as it provides the LLM with richer context from multiple related nodes, rather than isolated text attributes and often predicting the dominant category of a bundle is an easier and more reliable task for an LLM than classifying individual, potentially ambiguous nodes. Some theoretical results surrounding convergence was presented and empirical studies we conducted across ten TAG datasets with ablations. The reviewers had mixed response initially. While acknowledging interesting setup and extensive experiments, many important concerns were raised around handling of heterophily, too much dependence on initial embeddings, scalability to larger graphs, limited exploration on types of GNN, relevance of the theoretical results, and finally the significance of the work. We thank the authors and reviewers for engaging during the rebuttal period towards improving the paper: The new experiments on the large-scale ogbn-products dataset helped with scalability, experiments by injecting noise to initial embeddings showed its robustness, asking LLM itself to estimate the homophily ratio of a graph by querying it on sampled node pairs as well as using adaptive sampling handles some of heterophily concerns, and finally additional experiment on another GNN architecture GraphSAGE shows the generality of the method. Thus authors' rebuttal addressed many of the weaknesses pointed out by the reviewers and while some limitations remain, the paper makes a concrete contribution of interest to the community.
Decision: Accept (poster)