Glance for Context: Learning When to Leverage LLMs for Node-Aware GNN-LLM Fusion
Authors: Donald Loveland
Yao-An Yang
Danai Koutra
Venue: ICLR 2026 Poster (2026)
Keywords: GNN
Graph Learning
GNN-LLM
Homophily
Heterophily
Primary area: learning on graphs and other geometries & topologies
TL;DR: We show GNNs and LLMs excel on different structures and train a selective router to query the LLM for nodes that GNNs are likely to struggle on.

Abstract: Learning on text-attributed graphs has motivated the use of Large Language Models (LLMs) for graph learning. However, most fusion strategies are applied uniformly across all nodes and attain only small overall performance gains. We argue this result stems from aggregate metrics that obscure when LLMs provide benefit, inhibiting actionable signals for new strategies. In this work, we reframe LLM–GNN fusion around nodes where GNNs typically falter. We first show that performance can significantly differ between GNNs and LLMs, with each excelling on distinct structural patterns, such as local homophily. To leverage this finding, we propose **GLANCE** (**G**NN with **L**LM **A**ssistance for **N**eighbor- and **C**ontext-aware **E**mbeddings), a framework that invokes an LLM to refine a GNN's prediction. GLANCE employs a lightweight router that, given inexpensive per-node signals, decides whether to query the LLM. Since the LLM calls are non-differentiable, the router is trained with an advantage-based objective that compares the utility of querying the LLM against relying solely on the GNN. Across multiple benchmarks, GLANCE achieves the best performance balance across node subgroups, achieving significant gains on heterophilous nodes (up to +13\%) while simultaneously achieving top overall performance. Our findings highlight the value of adaptive, node-aware GNN-LLM architectures, where selectively invoking the LLM enables scalable deployment on large graphs without incurring high computational costs.

Published as a conference paper at ICLR 2026
GLANCE FOR CONTEXT: LEARNING WHEN TO LEVERAGE LLMS FOR NODE-AWARS GNN-LLM FUSION
Donald Loveland, Yao-An Yang & Danai Koutra
Department of Computer Science and Engineering University of Michigan
Ann Arbor, MI, USA
{dlovelan, ayayang, dkoutra) ch. edu
ABSTRACT
Learning text-attributed graphs has motivated the use of Large Language Models (LLMs) for graph learning. However, most fusion strategies are applied uniformly across all nodes and attain only small overall performance gains. We argue this result stems from aggregate metrics that obscure when LLMs provide benefit, inhibiting actionable signals for new strategies. In this work, we reframe LLM GNN fusion around nodes where GNNs typically falter. We first show that performance can significantly differ between GNNs and LLMs, with each excelling distinct structural patterns, such local homophily. To leverage this finding, we propose GLANCE (GNN with LLM Assistance for Neighbor- and Context-aware Embeddings), framework that invokes an LLM to refine GNN's prediction. GLANCE employs a lightweight router that, given inexpensive per-node signals, decides whether query the LLM. Since the LLM calls are non-di non-differentiable. the router is trained with an advantage-bare compares the utility of querying the I.IM against relying solely on the GNN. Across multiple benchmarks, GLANCE achieves the best performance balance across node subgroups, achieving significant gains on heterophilous nodes (up to +13%) while simultaneously achieving top overall performance, Our findings highlight the value of adaptive, node-aware GNN-LLM architectures, where selectively invoking the LLM enables scalable deployment on large graphs without incurring high computational costs.
objective that
INTRODUCTION
Text is rarely consumed in isolation: scientific papers cite related work (Ciotti et al. 2015 Mccallum
et al. 2018), users browse descriptions of co-purchased e-commerce items (Jin et al. McAuley & Yang 2016 Ezeife & Karlapalepu 2023), and social media posts reply to one another (Wu et al.] 2017 Yang & Leskovee 2011). These interactions form text-attributed graphs (TAGs), where
al.
2024
nodes represent text and edges capture relationships, enabling joint reasoning over content and structure (Yang et al. 2015] Zhao et al. 2023). Historically, TAGs have been processed by feeding shallow text features, such as TF-ID static word embeddings (Stephen al. 2022 Mikolov et al.) 2013), into Graph Neural Networks (GNNs). Recently, the success of Large Language Models (LLMs) has motivated hybrid architectures that leverage LLMs for graph learning, combining LLMs' semantic reasoning with GNNs' structural learning (Chen et al. [0224a] Wang et al. 2025). However, the majority of hybrid systems typically apply single fusion strategy across all nodes in a graph, overlooking per-node variations in semantic quality and structural attributes (Wu et al. 2024). This uniform application of LLMs across graph can waste expensive LLM calls on nodes already well modeled by the GNN, producing poor accuracy-efficieng tradeoffs (Liu et al. 2025).
key weakness uniform fusion failure to leverage the complementary strengths GNNs and LLMss. GNNs tend to perform well when nodes have a high degree and their neighbors exhibit homophily (i.e., con- nected nodes share labels). (McPherson et al. 2001 Yan et al. 2022]. Yet. these properties do not typically hold in real-world TAGs (Ma et al. 2020 Loveland et al. 2024 Zhou et al. (020). While advanced GNN de- signs have attempted to address these concerns (Zhu et al. 2020 Abu-EElHaijj et al. 2019 Veličković et al. 2018), recent evidence shows they remain insufficient for handling these challenging structures (Loveland & Koutra 2025) Mao et al. 2023 Du et al. (022). On the other hand, LLMs exhibit strong generalization in
low-shot settings, making them well suited for the challenging nodes that degrade GNNs (Chen al. 2024a

---

Published as a conference paper at ICLR 2026
Peng al. 2024).
However, leveraging LLMs for graphs often leads to the distortion of structural relation-
ships due to serializing the graph into text (Liu al. 2023 Firooz et al. [022 Wang et al. (025). This limita-
tion is particularly detrimental for graphs govemee by simpler structural signals where the LLM may add un-
necessary complexity and even degrade performance relative to GNNs (Liu et al. 2025 Wang et al. 2025
While incremental performance gains with high computational costs make LLMs seem ill-suited for graph learning, we argue this stems not from inherent deficiencies, but how LLMs are used. Our hypothesis that gains in regions difficult for GNNs are offset by losses elsewhere. Thus, we focus on utilizing LLMs only where GNNs struggle. prioritizing structurally difficult nodes whose errors are often masked in aggregate metries. This emphasis promotes equity across the graph, countering harmful GNN inductive biases (Wang et al. (022) Agaraal et al. [021]. This leads us to our core research question: How, and for which nodes, should we leverage I.L.s to complement and bolster GNNs?
To answer this question, we first identify signals that inform when GNNs vs. LLMs succeed. Since LLM routing has only recently been studied, we systematically assess existing methods (Qiao et al. [025 Jaiswal al. 2024] Chen et al. [2024b). finding highly variable performance, sometimes worse than random routing. Through a node-level analysis on local homophily and relative degree, two properties known to impact GNNs (Yan et al. 2022 Subramonian et al. 2024), we find GNNs excel in high-homophily. well-connected regions, while LLMs dominate as homophily or degree decreases. Notably, local homophily emerges strong predictor of model advantage, offering principled prior for routing.
Motivated by this, we propose GLANCE (GNN with LLM Assistance for Neighbor- and Context-award Embeddings), fusion strategy that preserves GNN efficiency easy nodes while selectively leveraging LLMs on hard ones. GLANCE first encodes nodes with a pre-trained GNN, then passes lightweight routing features a cost-aware policy that decides whether to query the LLM figuratively "glancing" at the LLM for additional context. For routed nodes, LLM embeddings are fused with GNN embeddings via small refiner head. To train the non-differentiable router, we introduce an advantage-brus baseed strategy that rewards beneficial queries to the LLM. We find that GLANCE is able to consistently outperform previous benchmarks, producing more robust predictions across the spectrum of homophily levels. This ability to achieve higher performance while reducing LLM usage, and thus computational cost, highlights the value of node-aware GNN-LLM fusion when scaling to large, diverse TAGs. Our contributions are below:
We provide systematic look LLM routing and show that current heuristics are brittle. By stratifying nodes by different properties, we reveal the complementary performance of GNNs and LLMs over local homophily and identify for which nodes LLMs are beneficial.
Current Limitations: Which Nodes to Route.
New GNN-LLM Method: How to Select Nodes. We introduce GLANCE, cost-aware framework that learns when to query the LLM, minimizing unnecessary LLM calls and preserving scalability.
Comprehensive Empirical Analysis. Across four diverse TAG datasets, GLANCE consistently outper- forms state-of-the-art GNNs and GNN-LLM hybrids, yielding robust predictions across homophily levels with gains of up + +13.0% on heterophilous pockets and overall performance gains of up to +0.9%.
2 RELATED WORK
We briefly discuss the most relevant work here, and provide more details in Appendix A
Static GNN-LLM Fusion. Prior work largely follows two paradigms: I.L.Ms-as-Enhancen and
SiMss-as-Predicton (Chen et al. 2024a Li et al. 2024). Enhancer methods enrich node features with
text embeddings or external knowledge (Wu et al. 2024 He et al. 2024), while predictor methods replace
the GNN with the LLM, directly classifying serialized graph inputs (Wu et al. 2024] Wang et al. (025).
LLM-as-Predictte methods can overcome GNN biases, but struggle with topology encoding and prompt
length (Firooz et al. (202)). Other fusion strategies draw from Mixture-of -fxxture-ff-Exeerss (MoE) (Cai et al. 2025
Wang et al.] 2023), but have not explored mixing GNNs and LLMs. Recent work has considered L.MSs to route among GNNs (Jiang & Luo| 2025). yet this still confines performance to GNN biases. In contrast, our approach adopts a selective paradigm, invoking the L.LM where it is expected to help, combining the strengths of both models.
Adaptive GNN-LLM Fusion. A challenge for LLM GNN models is inference cost, especially when LLMs are used in the forward pass for all nodes et 2024). While cost reduction via sampling distil- lation helps (Fang et al. [2023 Chen et al. 2024a)) scalability remains limited by universal LLM calls. Only few recent works explore adaptive GNN-LLMM fusion, where the L.M is invoked selectively. E-LLaGNN
2

---

2008a). Published as a conference paper at ICLR 2026
routes nodes with fixed heuristics (e.g., degree, centrality) (Jaswwal al. 2024). but requires manual tuning. LOGIN uses GNN uncertainty to rewire difficult nodes (Qiao et al. 2025) ) though this can erase useful het- erophilous links (Luan et al. 202T). LLM-GNN uses clustering density as proxy for difficulty (Chen al., 2024b). Our approach differs by: (1) systematically identifying structural properties predictive of L.LM henefit, learning to route without preset thresholds; (2) preserving graph structure rather than editing it; and (3) directly training router under non-differentiable queries, rather than altering data to create influence.
3 PRELIMINARIES
TAG is defined as G (V,E,T,Y), where is the set of n n= | nodes, CVVVVV iss the set of edges, T= [tt.)ee y is the text associated with each node u, and Y (ye Jev is the set of node labels. The task learn model s: (G.T) - that predicts node labels ye from both structure and text. Next we outline the strategies to parameterize and learn v/.
Graph Neural Networks. Node representations are learned by aggregating messages from neighbors by: h ((-1) AGGGRGGTT(())
(PDATE
Ve - arggmax
for node at layer l. Additionally, h, (0) =x, (a feature vector from (()) denotes the neighbors of node u, and AGGREGATE and UPDATE define the GNN's operations. In practice, AGGREGATE is a permutation-invariiae function (e.g. sum or mean) and UPDATE is multi- perceptron (MLP). After layers, the prediction is derived from an MLP head. Despite their success, GNNs can struggle on certain structural pattems, such as low degree and heterophily (Loveland et al.) (0244 Mao al.) [0033] Yan et al.) 2022). motivating architectrres with higher-order aggregation, residual connections, and adaptive message passing (Chen et al. 2020 Zhu et al.) 2020). We leverage GNNs that adopt these designs later our analysis.
Large Language Models. Let (-) denote a pre-trained language model. Given a node e with text t. and optionally neighbor attributes {tu : u (N()).. the LLM can either (i) generate an embedding Ze LLLMLL (Pc) from prompt Pe. optionally including neighborhood context, or (ii) directly predict the label d.- Lee (P.). Embedding LLMs allow the output 2g to be easily combined with a GNN's output or other downstream classifiers, but require prediction head. In contrast, direct prediction avoids this head but risks hallucinated labels and often still needs fine-tuning. this work, we adopt the embedding setting, which supports seamless integration with GNN representations.
4 WHICH NODES To ROUTE: CURRENT LIMITATIONS & OPPORTUNITIES
We begin by examining which nodes should be routed, reviewing prior routing strategies and highlighting their limitations. Then, building insights from the GNN literature, particularly the challenges with GNNs applied to low-degree nodes and heterophilous neighborhoodss (Yan et al.) 2022 Tang et al. 2020). show that LLMs can complement GNNs under these conditions. Ultimately, find that homophily emerges as a strong indicator for LLM benefit, offering a promising opportunity to improve graph leaming.
4.1 LIMITATIONS OF CURRENT ROUTING STRATEGIES
Based on previous work, we analyze strategies to route nodes an LLM. We consider node degree d, (Jaiswal et al. (2024) clustering density (C-density) (Chen et al.) 2024b). and uncertainty from dropout (Qiao et al. (2255) For each strategy, we route the top-k% of nodes a fine-tuned LLM under three criteria: (i) low degree, (ii) low density, and (iii) high uncertainty.
Experimental Setup. We evaluate on Cora (Mccallum et al. 2000 Pubmed (Sen et al. and Arxiv23 (He et al.) (2024) with processing details in Section and GCNII, using the dataset's original features (denoted as "Std.") and enhanced features (denoted as "Enh.") generated via Qwen3-8B. GCN serves as a traditional baseline, while GCNII introduces new designs to better handle over-smoothing and other challenging graph properties. Together, they provide contrast between simple message-pasee passing framework and stronger state-of-the-art backbone. For LLM-as- preedicto,, we fine-tune Qwen3-8B 3BB with an MLP head for node classification. LLM prompting and training for both and hyperparameters for the LLMs and GNNs are provided in Section C For the routing strategies in Table we freeze the GNN and L.MSs and route using the metric.
D We train two backbones, GCN
and LLM-as as-Predictor
given in Section B Training details
3

---

Published as a conference paper at ICLR 2026
We assess heuristics with net correction score (NCS), capturing the benefit of routing. For routed set R. let WC be the nodes wrong under the GNN but became correct after routing to the LLM, and CW be the set of nodes were correct but became wrong. We define NCS= = (I NCS=1 1 means the L.M fixes all routed nodes and -1 means the LLM harmed every routed node.
where
Findings. Table 1 highlights the dif- ficulty to identify single routing sig- nal that generalizes across datasets. On Pubmed and Arxiv23, uncertainty appears promising, achieving the best NCS in ev- cry setting. Yet on Cora, the same strat- egy consistently produces negative NCS, often performing worse than random router. Other heuristics such as degree and C-density similarly fluctuate in effec- tiveness, with no strategy performing ro- bustly across all datasets. Together, these results underscore the limitations of static heuristics: while they succeed in iso- lated cases, their performance is dataset-dependent. This motivates the need for more principled and transferable routing signal, rather than manually chosen rules.
Table 1: NCS for C-density, d., and uncertainty, as compared to a random router. higher is better. The number of routed nodes is in parenthesis. The highest NCS score is bold for each setting.
Cora
Pubueed
Arsiv23
10% 15%
20%
10%
15%
20%
10%
15%
20% (978)
Routtng Strut.
(68)
(102)
(136)
(493)
(740)
(986)
(489)
(7344
Random
-0.02 -0.06
-0.04
0.06
0.05
0.04
0.05
0.04
0.04
GCN
C-density
-0.02
40.03
-0.03
0.05
0.06
0.05
0.07
0.07
0.06
Emnh
Degree
.0.44
.0.01
-0.02
0.04
0.94
0.04
0.04
0.05
0.05
Uncertainty
-0.09
-0.03
-0.01
0.20
0.18
0.17
0.15
0.13
0.13
Random
0.01
40.01
0.01
0.01
0.01
0.00
0.00
GCNI
C-density
0.00
-0.03
-0.03
0.02
0.02
0,02
0,03
0.03
0.03
Enh.
Degree
0.03
00.03
0.002
0.03
0.03
0.03
.0.03
.0.02
.0.01
Uncertiinty
-0.04
-0.02
-0.03
0.09
0.08
0.08
0.05
0.04
0.05
4.2 STRUCTURAL OPPORTUNITIES FOR ROUTING
Given that heuristic quality can be datasct-dependent we aim to find a routing signal that aligns with the strengths of GNNs and L.M We first analyze metrics known to degrade GNN performance and then utilize these to route, showing that homophily is strong indicator for when L.LMs improve performance.
4.2.1 ON THE COMPLEMENTARY CAPABILITIES OF LLMs AND GNNS
To characterize the unique benefits of LLMs and GNNs, we take the models from Table and perform stratified analysis using relative degree d, and local homophily h.g When de > 1, e tends to have higher degree than its neighbors, and lower degree otherwise. Low h. indicates is heterophilous. Mathematically,
d
[c)i EuENTO)
dut1 1
by = [() EveN( [i -vo]
Both properties are known to influence GNNs (Yan et al. 2022 Subramonian et al., 2024). but have received less attention for LLM-graph reasoning. We compare models actoss these properties understand how each paradigm responds.
Findings. In Figure D we study per- formance for groups of nodes strat- ified by hy and dy. We find that LLM models tend to produce sig- nificantly higher performance over heterophilous and low-degree nodes, e.g., achieving up to 20.4% per- formance increase compared to the next best model, GCNII with LLM
Cora
Andi23
1150
8800
1258
66N (X))
Todd:
(b
500
2000
LMM
4.80 A00
d.00
Accuracy (haetbee Degree Strelted)
Homophayy Bn
Homophtly
enhancement on Cora. Additional re-
sults given for Pubmed in Section E.4 with similar trends. In Figure 4 (Sec- tion (.5). we further find that ho- mophily and degree can interplay with
1000
4830
2000
3000
one another, where performance dif- ferences can reach upwards of 30.1% between subpopulations stratified by both degree and local homophily. With these findings, we next study if ho- mophily can be useful for routing.
13 100
d20
1000
Relotvve Degice 8n
Refative Degroe 8in
Figure 1: Stratified performance for ha (top) and d, (bottom). Bars denote property distributions (right y-axis). Enhanced GNNs can henefit heterophilous & low degree nodes, but M.M offer further improvements.
4

---

Published as a conference paper at ICLR 2026
Table 2: (Left) NCS for bas Ni., and de routing, where higher is better. Homophily yields high NCS, with estimated and true variants improving as k increases. (Right) Mean rank across routing strategies, lower is better. Bold is best out of the label-free methods; we note that h. cannot be used during inference due requiring access labels.
Cora
Pubmed
Arxiv23
Avg. Ranking
Label Free
For Inference
Strat.
10%
15%
20%
10%
15%
20%
10%
15%
20%
Random
4.50
ky
0.24
0.11
0.05
0.29
0.30
0.26
0.15
0.14
0.15
C-densily
4.14
GCN Enh.
-0.07
-0.04
-0.04
0.20
0.20
0.19
0.14
0.14
0.15
Degree
4.35
da -0.07
-0.07
-0.06
0.06
0.04
0.04
0.03
0.05
0.03
Uncertainty
3.28
d
hy
0.09
0.03
0.00
0.10
0.11
0.09
0.04
0.05
0.06
GCNT
Ry -0.06
-0.03
-0.03
0.07
0.07
0.06
0.04
0.04
0.05
3.22
-0.06
-0.07
-0.05
0.03
0.02
-0.03
-0.01
-0.01
hv
1.03
X
4.2.2 ROUTING WITH LOCAL. HOMOPHILY
Unlike degree, which can be derived directly from the graph structure, homophily depends class labels that are not accessible during training. This makes it challenging to directly exploit the performance gains LLMs provide on heterophilous nodes (Figure T Motivated by prior work on classifying edges as homophilous or heterophilous (Du et al. 2022 Wu et al. 2024). we extend this idea to estimate node-level homophily score. Concretely, we train MLP Q to predict node labels, d. = argmax Q(x.). ). and use these predictions to compute an estimated local homophily, ht (o(e) EmeN(,) (w) [ik. ūu]. This proxy enables us to leverage the benefits of homophily for routing without requiring ground truth labels. We employ an MLP to avoid the structural biases that GNNs impose during homophily estimation (Zhu al. 202T).
Findings. Table [ shows that homophily is a useful routing signal. First, we find that true local homophily, hys attains the highest NCS in most settings, establishing an upper bound for structure-aware routing. More importantly, our label-free homophily estimate closely tracks hy and typically matches or surpasses other static heuristics. The average rankings in Table 2 corroborates this finding, where hg has the best mean rank for NCS score. Moreover, when h, is excluded from the ranking, our estimated homophily achieves the best average rank. Overall, homophily reliably identifies nodes that benefit from LLM assistance, and our label-free homophily proxy is a practical and effective prior for routing.
5 How TO ROUTE: ADAPTIVELY FUSING LLMS AND GNNS WITH GLANCE
To address the limitations of existing routing strategies, we propose GLANCE, a framework that adaptively fuses GNNs and L.Mss. Rather than relying on handcrafted rules, GLANCE employs lightweight router trained to decide, on a per-node basis, whether to invoke the LLM. This design ensures that the LLM is used only when it provides improvement to justify its cost. GLANCE is comprised of three components: (i) frozen GNN and LLM encoders, (ii) a trainable router using cheap features for routing, and (iii) a combiner that fuses structure and text embeddings into final prediction.
5.1 COMPONENTS OF GLANCE
We now detail the components of GLANCE, as depicted in Figure for routing. Then, we specify how the LLM generates new embeddings for routed nodes. Finally, we define the refinement process that merges the GNN and LLM information. As this is a non-differentiable pipeline, we include details on how the router is trained to encourage effective usage of the LLM.
2 We first outline how nodes chosen
5.1.1 STEP 1: GENERATING AND PROCESSING ROUTING FEATURES.
Routing Features. We begin by using a pre-trained GNN F to produce embedding zo(v) from the k-hop neighborhood of v. This embedding acts as the first signal to ensure the router can leverage the structural information of the neighborhood. We also use the GNN to derive uncertainty estimations the node, following a dropout strategy (Qiao et al. 22025). This uncertainty acts as a proxy signal for difficulty, allowing the router to take advantage of the relationships found in Table I Building leverage homophily, we utilize Q to attain a local homophily estimate. However, to increase expressivity, we compute the probability distribution over the classes, PQ.., and estimate a soft local homophily as:
the opportunity to
h. =PQ."
[N((()]
1
PQ. (w))(()
Σ
(1)
5

---

Published as a conference paper at ICLR 2026
Step Generote and Process Rousing Features
Stop 2. Use Pre tralied LLM Process Routed Neighborthodd
Text Attr intributed Graph (TAG)
Node Embedding
Ego Node
LLM (Qwenn-86)
TTEO
and Text:
Node Uncertaintty
Maaad
Hamophilly Estimation
Routev/
One Hop
MLP
a
Nodias
LLM (Owen-SB))
and Tesd:
Node Features
moone *
Two Hop
Griph with Testures
Degres
and Text:
Nodes
LUM (Owen--B 3B)
Batch of Refined Predictions
Row Node Text
Batch Predittii
GNN Embcddings for Rouned hores
ULM Emioddiigg Routed Nodes
Step Rofhe GNN predictions using LLM emocdstings
Figure 2: GLANCE Overview. Step I: GLANCE generates routing features to derive decision. Step 2: routed node's 's text is fed into the LLM to generate embedklings. Step 3: A routed node's 's GNN & LLM embeddings are used to refine predictions. For nodes not routed, the GNN prediction is used. Only the router and refiner MLP are trained.
This soft estimate provides the router with a measure of neighborhood alignment. While Table Sshlesssoows that fúg cannot definitively route on its own, we utilize this signal as a prior to bias the router toward heterophilous nodes, which we expect to be refined during training. While uncertainty was originally motivated to capture heterophily Qiao et al. (2025), we find this correlation to be weak (seen in Table 9 and use both metrics given they provide different, yet informative signals. Finally, include the original features and degree, enabling the routing of nodes with noisy features or insufficient neighborhood context.
Node Router.
We define router ı that takes as input the routing features above. denoted as a vector f,, and outputs probability of routing a, < [0..1], for node v, as a =((,) oo(w" f.). where larger values of a, indicate that the LLM should be queried for node v, while low values of ag indicates that the GNN prediction is sufficient for node v. Rather than applying an absolute threshold, GLANCE uses top-k strategy: for each mini-batch, the & nodes with the highest a, scores are routed to the LLM. This ensures a fixed query budget per batch and avoids the need to globally calibrate the router's probabilities.
5.1.2 STEP 2:
LLM TO PROCESS ROUTED NEIGHBORHOODS
For the routed node set R. the LLM encoder L is used to generate embeddings from serialized neighborhood prompts (t)) Ao E R, examples prompts given in Section B Rather than producing a single embedding in previous work (Wang et al. (0255), we generate embeddings at multiple structural levels: (1) the ego text l.., (2) the ego text with a sampled set of 1-hop neighbors {t, :wNi (v)),,
J. (3) the ego text with sampled set of 2-hop neighbors (tg Uf, u N2(1)) Each level serialized into separate prompts, encoded into embeddings ZL. ZL, 1
(v) LL (v), and concatenated to form the final embedding:
zz (v) (z,
This encoding preserves both ego and neighbor information, while ensuring the prompt length remains manageable. Additionally, this design aligns with the aggregation typically seen advanced GNNs (Zhu) al. 2020 Yan et al. (2022) Using LLM embeddings also offers computational advantages, as it avoids the costly generation step required by prior approaches (Qiao et al. 2025 Wu et al. 2024).
5.1.3 STEP 3: REFINE THE GNN PREDICTIONS USING LLM EMBEDDINGS.
For nodes not routed, we retain their original GNN predictions by feeding the GNN-based node embeddings into the original MLP head used during GNN training. For nodes that are routed, we want to utilize both the GNN and LLM embeddings to capture structure and contextual information. Thus, we define a refiner MLP C' that integrates the GNN and LLM embedding as:
PC = (

---

Published as a conference paper at ICLR 2026
where PC,e is the probability distribution over the classes for node This modular design is agnostic to a specific GNN or LLM backbone, enabling flexibility depending on the dataset or computational budget.
TRAINING OBJECTIVES
Because routing decisions are discrete and require prompt construction, routing weights cannot be directly learned via backpropagation. Below, we detail how we translate the routing signal into weight updates.
Counterfactual Comparison and Rewards. When routing to the LLM, we measure the value of invoking it by computing a counterfactual prediction using the GNN and measuring the difference in losses produced by the two computations. Specifically, with zc(#) and the GNN's pre-trained prediction head H:
PH. so.:=sttmx((((cc()
[GNM =- -bh 1ly 1y = (LLM)) = -V =kllopp.
where (.T.MM) is the loss when routing with the LLM, and (GNN)) is the counterfactual loss from the GNN. Note, the counterfactual is only computed when nodes routed the L.M during training, i.e. we do not compute an LLM-based counterfactual for nodes not routed. We then quantify the benefit of routing (or not routing) the LLM through the rewards:
(GNN) - -B, (GNN)
ag in top-k (LLM queried), ag not in top-k (LLM queried).
Tv
The first term captures the decrease loss provided by querying the LLM and the cost of invoking the LLM (hyperparameter B 0). Larger 3 imposes stricter penalties, discouraging the use of the LLM where doesn't 't provide benefit. A positive ra with LLM routing indicates that calling the LLM reduced prediction loss enough to offset its cost. When the LLM is not used, the reward is based on the GNN loss.
Training Objective. The router is optimized with a policy gradient-inspire loss, treating routing as contextual bandit problem and encouraging alignment with counterfactual advantage:
groute)) - (ooom((f.)) Hent (T..)]
While this objective is inspired by REINFORCE (Sutton et al. 1999). we employ deterministic top-k selection during training to ensure stable use the limited LLM query budget. The final objective jointly trains C' and through:
(pred) = 1aa (LM)) +(1-1(11111 1(a, E e
(GNN)
C
p(pred)
Avouter e prrott))
BEB
We use g(preed denote the prediction loss coming from either the LLM or GNN computation paths. Then, the first term of C optimizes predictive performance, while the second enforces cost-aware routing. For efficiency, both the GNN F and the LLM are kept frozen during training, training only C' and I.
EMPIRICAL ANALYSIS
We now empirically study GLANCE, demonstratng how selectively using LLMs can improve performance. We first analyze GLANCE's performance, showing that it achieves significantly more balanced performance compared to the baselines. Then, we provide a series of supplemental analyses to understand what GLANCE leams during training. Finally, we extend GLANCE larger TAGs demonstrate its scalability.

---

Published as a conference paper at ICLR 2026
Table 3:
Per-bin accuracy for homophily levels. Training Strategies: O for Original features. E for Enhanced features,
and L for LOGIN. Bold denotes best method. GLANCE achieves the strongest and most balanced perfornance across local homophily levels, as evidenced by its lowest average rank across datasets and homophily bins (righumost column).
Cora
Pubmed
Arxiv23
rank
0,00-0.25 0,25-150
0.50-0.55
0,75-100
0,00-0.25
0,25-0.50
0.30-0.5 0.75
0,75-1,00
0,00-0,25
0.25-0.50
0,50-4.75
0,75-1.00
0 1877667
3662
71.139
98551:2
48244-422
100:11
83.111111
97.2+011
25.14:0A
47. 47.1=220
77,0456
93.02299
103
GCN
1799669
49.6
785+333
98.7+03
55.9+3.7
8188
849008
97.9+0.1
27.8+300
46.4220
79.1+-44
95.1+21
L 17.6db4.3
44.7220
78.9234
978200
47.02077
45.522411
13.1216
98.1:0022
24940.
57.611.0
64.52111
8322:0.1
10.4
0 21.1229
45.213
76.22488
97.9:255
584422
16.111
85.32206
966-02
32.60.3
$33400
8055077
939:044
86
SAGE
3061669
48.616611
754433
96.1106
695:2.0
89.2:005
97440.1
3631004
56.11:044
81.2107
918.103
6.
2688:226 43.6#500
75/4338
97.8404
63.1+:13
69.6
08.5=009
97.6:004
37.0+:27
51.613.1
76.0:111
89:::10
8.2
20.0755 41.7+377
83.1200
989203
54.4499
$15+6611
8588+03
97.0+011
3199111
49843.1
7855+444
92.9-222
8.9
GONI
B $2.0779
535535
97.1009
09742211
14:44:6
889209
074404
41.612
580=27
13.2419
9484077
4.7
33.4199 508824
77.8:-111
96.6:11
71.0115
68.72211
88.61111
97.1
4666::66
62.0=0.7
81,2409
92.5+09
FAGUN
27.1472
509448
83.2:44
98.9:05
097:22
65.3+32
885212
97.1004
378+25
564227
82.9+11
94-641
GGCN
R 27.7284
502199
71
958-099
68
660228
09440.
91.2401
42.340055
62.12044
85.0:20
953:10.3
53
GBK-GNN
28.6.148
510:2.1
78
96.7088
675519
679143
879.17
979100
38.7109
1.8 83.8.12
94310.11
58
GLANCE
46.41224 54.4::66
79.7=00,9
98.1:002
7L5105
65.9:044
89.4004
97.710.1
45.2:0077
63.2::02
85.9.0033
953:1000
2.4
6.1 EXPERIMENTAL SETUP
Datasets, We first evaluate across Cora (Mocallum et al. 2000), Pubmed (Sen et al. 2008b), Arxiv23 (He
al. 2024), studying overall and stratified performance, as well as performing numerous sensitivity and ablation analyses. Then, to demonstrate GLANCE's scalability, we evaluate on two large-scale datasets, Arxiv-Year (Lim al. 202T) and DGB-Products with details provided in Section D For each dataset, we
follow the same training set up as outlined in the Section
Baselines & Training, We compare GLANCE a series of state-of-the-art baselines for scalable LLM- GNN fusion. We start by considering standard GNN backbones, including GCN (Kipf' & Welling 2017). GraphSAGE (Hamilton et al., 2017). and GCNII (Chen et al. (020).. We study their performance over three settings, original features, L.LM-enhanced features, and T. LOGIN-filteeed graphs. As GLANCE's core design is to bolster challenging heterophilous nodes, we also include high-performing heterophilous GNNs, including FAGCN, GGCN, and GBK-GNN. To imbue them with LLM knowledge, we utilize LLM- enhanced features. We do not apply LOGIN on these models as they are designed to utilize heterophily, whereas LOGIN's design intends to remove it. Across all baselines, protocols, and text encoders. Training details and hyperparameters are provided in Section C Unless otherwise stated, we route the top 12 nodes per batch of 32 nodes for GLANCE for reported metrics. For OGB-Products, we evaluate against the top 3 model combinations found in our initial study.
use identical data splits, training
6.2 CAN GLANCE ACHIEVE THE BENEFITS OF LLMS AND GNNs IN ONE MODEL.?
As shown in Figure I selectively using the LLM on challenging nodes can enhance GNN performance. To evaluate GLANCE's capabilities, we report both overall accuracy (Table 4 and strati- fied results (Table GLANCE attains the best performance on all three datasets, achieving 89.5% on Cora, 92.6% on Pubmed, and 82.1 1% on Arxiv23, leading to an average gain of +0.5% over the next best model. Notably, GLANCE requires signifi- cantly fewer LLM queries to attain similar performance gains found in previous work (Wang al. 2025 Chen et al. 20244). GLANCE also consistently outperforms LLM-enhanced and LOGIN baselines, often by several percent, as well as GNNs de- signed for heterophily. From stratified perspective, the largest improvements emerge on heterophilous nodes, where GLANCE delivers +13% Cora and +0.5% on Pubmed, Importantly, when averaging across bins, GLANCE achieves the strongest overall balance with an average rank of 2.4, compared to 4.7 for the next best. Together, these findings demonstrate that GLANCE's routing reliably leverages the LLM where it is most beneficial while preserving the strengths of the GNN.
Table 4: Acc for 3 runs. Bold best
Cora
Pubmed
Arsiv23
O 87.1
88 18.16:066
74,2+22
GCN
E 87.9413
8991::66
76.24:1.2
868100
8861002
65.7:1000
86,910
89.74:044
77.2#:02
SAGE
B 86.41.11
922210.4
78.1::0.1
86.4103
91.8:..11
74.71:155
O 88.41088
89.1:10.6
75.91:21
GCNIT
E. 83.7
92.111022
80.21077
L 86.820.7
91.9:0.11
79.71:05
FAGCN
E 89,4#05
91884002
79.2+:1.
GGCN
E 85.4:09
92.2::022
8122405
GBK-GNN 86.6110
92.110.3 79.5104
GLANCE
89.5:04
92.610011 82.1::011
8

---

Published as a conference paper at ICLR 2026
Properties of Routed Nodes. In Figure analyze the local homophily of routed nodes, splitting them into benefited (GNN wrong, LLM right) and non-benefited sets. Across datasets, we observe substantial mass at low homophily, showing that
Cora
1.00
0.75
GLANCE preferentially routes heterophilous nodes. Crucially, the benefit distribution is skewed toward low homophily, indicat- ing that routed heterophilous nodes deliver performance gains. This supports our hypothesis that heterophily captures GNN failure modes where LLMs are most valuable, and shows that these nodes drive GLANCE's improvement. We also high- light that the median beneficial homophily value is not identical across datasets, e.g., on Arxiv23 the largest gains occur around he0..5. Thus, a single homophily threshold that only routes heterophilous nodes would be ineffective for maximizing per- formance, and additional context is needed.
0.50
0.25
0.00
Pubmed
Local Homophily
1.00
0.75
0.50
0.25
0.00
Arxiv23
1.00
we examine per- formance under varying routing budgets. For a batch size of 32, we evaluate K E {8, 12, 16]. We find that increasing K tends lead steady gains on heterophilous nodes, while leav- ing performance on the homophilous nodes mostly unchanged. On Pubmed and Arxiv23, accuracy on h, 0.25 improves by +3.4% for 8- 12, and by another +3.0% for K - 12 16. On Cora, performance initially dips for K 8 12, then sig- nificantly improves by + +2.3% when increasing to K = 16. For he > 0.75, GLANCE's accuracy drops by a marginal -0.06%
Sensitivity to K. Figure (Section F.1
0.75
0.50
0.25
0.00
Benaticial (G:K, Li)) Not Benefirial
Figure 3: Local homophily for routed nodes, split by benefit. Blue lines denote median.
on average across datasets, underscoring GLANCE's stability in this regime. Importantly, we also observe that routing too many nodes can harm accuracy (as seen Table [0] full-batch routing). as easy nodes well modeled by the GNN can become misclassified when the LLM is unnecessarily used. Together, these findings demonstrate that GLANCE routes efficiently, mitigating both the performance degradation and computational overhead of indiscriminate LLM usage.
6.3 ROUTER: LEARNED PROPERTIES AND ROBUSTNESS
Routing Feature Ablation. To assess the value of the routing features, we disable each feature one at a time and train new GLANCE models, measuring Accai Acciul (Figures in Section F3 Averaged over all features, accuracy drops on each dataset, with changes of -0.38% on Cora, 1.07% on Pubmed, and -0.65% on Arxiv23. More importantly, when ablating the homophily feature, find significant drops when h, < 0.5, with changes of -6.5% for Cora, -6. -6.3% for Pubmed, and 22.% for Arxiv23. We also observe subtle losses when h, 0.5 as the homophily signal is removed, where Cora drops by -0.34% Pubmed drops by -0.15%, and Arxiv23 drops by -0.3%. These drops indicate that homophily helps the router avoid misrouting the nodes easy for GNNs. Together, these ablations show that GLANCE achieves peak performance when all features are available, with the largest gains on heterophilous nodes.
6.4 LARGE-SCALE LEARNING WITH GLANCE
Datasets. GLANCE's selective utilization of the LLM enables the processing of large TAGs by only applying the LLM to nodes that benefit. Now that we have demonstrated the effectiveness of GLANCE. we study the scalability by applying GLANCE two large benchmark datasets, Arxiv-Yess Year (Lim et 202T) and OGB-Products (Hu et al. 2020). Details for these datasets can be found in Section D Notably, Arxiy- Axar and CGB-Products are multiple orders of magnitude larger than the datasets typically studied
in GNN-LLM papers (Qiao et al., 2025 Wu et al. 2024 Wang al.] (025). Furthermore, to motivate GLANCE's scalability, we show in Figure that the routing and refinement components GLANCE create very little overhead, enabling to handle larger datasets when querying less nodes with the LLM.
Results. In Table 5 provide per-bin and overall accuracy for the top 3 average rank models from Table and compare them to GLANCE, Given these datasets are significantly larger than the previous datasets, we utilize query rate of ~ 1.6% (K= with batch size 64). We find that GLANCE outperforms

---

Published as a conference paper at ICLR 2026
Table 5: Per-bin accuracy for homophily levels and overall performance on larger datasets, We compare GLANCE the next best ranking models from Table using the enhanced feature strategy (E), OOM GGCN discussed Section C.I
Bold denotes best. We find the benefits carry to large-scale datasets with higher overall performance.
Year
OGB-Products
0.000::5
0.25-0.50
0.50-0.75
0.75-1.00
Overall
0.000:255
0.25-0.50
0.50-0.75
0,75-1.00
Overall
GONI
42.7::0.0
51.1100
54.-000
72440011
49.6:0.1
29.0::0.1
4399-0.1
69.3-00
91,910011
81.820.1
FAGCN
B
3600+009
4500=0.
527=02
72.0+05
44.3005
29
42.3+133
64.7=111
18
78.6=099
GGCN
368::00.5
46.3+066
51.6114
69.31122
44.620.1
OOM
OOM
0OM
0OM
0OM
GLANCE
43.1:0.11
513::0.1
$4.2=00
72.1
49.8:0.1
29.7:20.3
44.4:0.0
69.9-0.11
92.3-10.00
82.3:0.1
the best baselines while achieving strong performance across the spectrum of homophily levels. These results demonstrate that selective LLM queries, even at very low query rates, can enable gains and supplement GNNs where they struggle. Given the results we also expect this performance can become even better with a larger compute budget. Moreover, for applications that necessitate better performance heterophilous pockets of nodes, GLANCE able to offer key benefits, regardless of dataset size, that are otherwise inaccessible with standard GNN architectures.
above with regards to sensitivity to K,
7 CONCLUSION
In this work focused on how LLMs can be selectively utilized to bolster GNNs, an understudied area in the LLM-GNN literature. Through detailed analysis of structural properties, we showed that homophily provides a reliable signal for routing, capturing where GNNs tend to fail and LLMs excel. To operationalize this finding, we proposed GLANCE, a cost-aware fusion strategy that learns when to route, calling the LLM difficult nodes. Empirical results across multiple benchmarks demonstrate that GLANCE achieves well-balanced performance while selectively invoking the LLM only when useful. Our findings argue for structure-aware routing foundation for future work efficient GNN-LLM integration.
ACKNOWLEDGMENTS
The material in this work is supported by the National Science Foundation under IIS 2504090, IIS 2212143, and CAREER Grant No.IIS 1845491. Any opinions, findings, and conclusions or recommendations expressed in this material are those the author(s) and do not necessarily reflect the views of the National Science Foundation other funding parties.
REFERENCES
Sami Abu-al-Haija, Bryan Perozzi, Amol Kapoor, Hrayr Harutyunyan, Nazanin Alipourfard, Kristina Lerman, Greg Ver Steeg, and Aram Galstyan. Mixhop: Higher-order graph convolution architectures via sparsified neighborhood mixing. In International Conference on Machine Learning (ICMI), 2019.
Chirag Agarwal, Himabindu Lakkaraju, and Marinka Zitmik. Towards a unified framework for fair and stable graph representation learning 2021. URLL/ttts://axxvv.ooo wwwp:iii.rrr//aa. hwtp://b//2//
2212.3.386 13186
Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. A survey on mixture of experts in large language models. IEEE Transactions Knowledge and Data Engineering, pp. 1-20. 2025. ISSN 3865. doi: 10.1109/tkd..2225..32 3554002 URL BRLLttt::/// / /xx doi. or..0.o0/00..0000
TKDE 2025. 3554028
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolu- tional networks. In International Conference on Machine Learning. 2020.
Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang. Dawei Yin, Wenqi Fan, Hui Liu, and Jiliang Tang. Exploring the potential of large language models (Ilms) in learning on graphs, 2024a. URL htttp::///a.ii......// org abs 2207. 03393
Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, and Jiliang Tang. Label-free node classification on graphs with large language models (Ilms), 2024b. URL https://arxiv.orgg/
10

---

Published as a conference paper at ICLR 2026
Valerio Ciotti, Moreno Bonaventura, Vincenzo Nicosia, Pietro Panzarasa, and Vito Latora. Homophily and missing links in citation networks. EPJ Data Science, 5, 11 2015.
Arman Cohan, Franck Demoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. A discourse-aware attention model for abstractive summarization of long documents. In Marilyn Walker, Heng Ji, and Amanda Stent (e Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Vol- ume 2 (Short Papers), pp. 615-622 621, New Orlcans, Louisiana, June 2018, Association for Computational Linguistics. doi: 10. 18653/v 1/N18- 2097. URL httpt::t////t a anthoooohoyyy .rg/N18 2097/
Lun Du, Xiaozhou Shi, Qiang Fu, Xiaojun Ma, Hengyu Liu, Shi Han, and Dongmei Zhang. Gbk-gnn: Gated bi-kernel graph neural networks for modeling both homophily and heterophily. In Proceedings of the ACM Web Conference 2022, pp. 1550-1558, 2022.
Christie I. Ezeife and Hemni Karlapalepu. A survey of sequential pattern based e-commerce recom- mendation systems. Algoritiums, 16(1)) 2023. ISSN 1999-4893. doi: 10.3390/a16100667. URL https://wwer 40t/:///99999999///1////////
Taoran Fang, Yunchao Mercer Zhang, Yang Yang. Chumping Wang, and Lei CHEN. Universal prompt tuning for graph neural networks. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023. BtLLLtttsss//peeeeviert net/ forum?id=0LmWWihh
Hamed Firooz, Maziar Sanjabi, Wenlong Jiang, and Xiaoling Zhai. Lost-in-distance: Impact of contextual proximity on IIm performance in graph tasks, 2025. URL http:p://axxxxvoorrrraa/////////
01 985
Will Hamilton, Zhitao Ying. and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, 2017.
Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Harnessing explanations: LIm-to-Im interpreter for enhanced text-attributed graph representation learning, 2024.
BtRLtttss://axttt wrg. 2305. 19533
Weihua Hu, Mathias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: datasets for machine learning graphs, In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 7781713829546
Ajay Jaiswal, Nurendra Choudhary, Ravinarayana Adkathimar, Muthu P. Alagappan, Gaurush Hiranandani, Ying Ding, Zhangyang Wang. Edward W Huang, and Karthik Subbian. All against some: Efficient integration of large language models for message passing in graph neural networks, 2024. URL https://arrx wwwpaaawwi....gaaaaaaaa http:////222222.. 14996
Dapeng Jiang and Xiao Luo. Marrying LLMs with dynamic forecasting: A graph mixture-of-expert perspective. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.). Findings of the Association for Compu- tational Lingmistics: NAACL 2025, pp. 396-410, Albuquerque, New Mexico, April 2025, Association for Computational Linguistics. ISBN 979-8-899 89176-976671117 195-7. doi: 10. 18651//v112225 mandings-naacl.. org/ 2025. f findiniiiii Lindings-naace /44
bhttps://aclancee
Wei Jin, Haitao Mao, Zheng Li. Haoming Jiang, Chen Luo, Hongzhi Wen, Haoyu Han, Hanqing Lu, Zhengyang Wang. Ruirui Li, et al. Amazon-m2: A multilingual multi-locale shopping session dataset for recommendation and text generation. Advances in Neural Information Processing Systems, 36, 2024.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. A survey of graph meets large language model: Progress and future directions, 2024. URL https
//axxiv. orggo///g/oo.....o.oo... absa 231 12399
Derek Lim, Felix Hohne, Xiuyu Li, Sijia Linda Huang, Vaishnavi Gupta, Omkar Bhalerao, and Ser Nam Lim. Large scale learning on non- non-hooophilous graphs: New benchmarks and strong simple methods. In NewrlPS, volume 34. Curran Associates, Inc., 2021.

---

Published as a conference paper at ICLR 2026
Jiacheng Lin, Kun Qian, Haoyu Han, Nurendra Choudhary, Tianxin Wei, Zhongruo Wang, Sahika Genc, Edward W Huang, Sheng Wang, Karthik Subbian, Danai Koutra, and Jimeng Sun. Gt2vec: Large language models as multi-modal encoders for text and graph-structured data, 2025. URL
https://arxivvorgg
2410. 11235
Jiajin Liu, Dongzhe Fan, Jiacheng Shen, Chuanhao Ji, Daochen Zha, and Qiaoyu Tan. Graph-mlim: Hamessing multimodal large language models for multimodal graph learning, 2025. URL https
//arxiv.org/ abs/ 2506. 10282
Nelson Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and
Percy Liang. Lost in the middle: How language models use long contexts, 2023. URL https
//arxiv.org abs/ 2307 03172
Donald Loveland and Danai Koutra. Unveiling the impact of local homophily on gnn fairnsss: In-depth analysis and new benchmarks. SDM, 2025.
Donald Loveland, Jiong Zhu, Mark Heimann, Benjamin Fish, Michael T Schaub, and Danai Koutra. On performance discrepancies across local homophily levels in graph neural networks. In LoG, volume 231. PMLR, 27-30 Nov 2024.
Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao-Wen Chang, and Doina Precup. Is heterophily real nightmare for graph neural networks to do node classification?, 2021.
abs 0564415641111111
Jiaqi Ma. Shuangrui Ding, and Qiaozhu Mei. Towards more practical adversarial attacks on graph neural networks. Advances in neural information processing systems, 2020.
Haitao Mao, Zhikai Chen, Wei Jin, Haoyu Han, Yao Ma, Tong Zhao, Neil Shah, and Jiliang Tang. Demystifying structural disparity in graph neural networks: Can one size fit all? NeurIPS, volume 36, 2023.
Julian McAuley and Alex Yang. Addressing complex and subjective product-related queries with customer reviews. In Proceedings of the 25th International Conference on World Wide Web, WWW '1 pp. Republic and Canton of Geneva, CHE, 2016. International World Wide Web Conferences Steering Committee. ISBN 9781450341431.
doi: 10. 10.14552872427.2888444
https///:://do
org/
724277 2883044
Andrew Mccallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2), 11 2000.
Miller McPherson, Lynn Smith-Lovin, and James M Cook. Birds of a feather: Homophily in social networks. Annual review of sociology, 2001.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean, Efficient estimation of word representations in vector space, 2013. URL (Rt/httss///axxi org/ wwwp/r/g/////////////.///eeeeee/ httpgg/////111133//3 11111111111711111
Jingyu Peng. Qi Liu, Linan Yue, Zaixi Zhang, Kai Zhang, and Yunhao Sha. Towards few-shot self- explaining graph neural networks, 2024. URL htt.lhtts://// //arxiv////ivvvio org/ abs/ / 2400 07340
Yiran Qiao, Xiang Ao, Yang Liu, Jiarong Xu, Xiaoqian Sun, and Qing He. Login: A large language model consulted graph neural network training framework. Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM "25, pp. 232-241, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400013293. doi: 10.1145/7005511370044 URI.
https://doi.. wttp://...1../////.1...1 3701377111111 7011551.1100302000 3703488
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. Al Magazine, 29(3):33 Sep. 2008a. doi: 10. 1609 v29i3.2157. URL battps://oji aaai org/ httgg//aaaaaazineeeeeea wiw.eneeeiinxxeeeeee hww.ad/ww.pp//immaaziiaii. Neticle/view/21 2157
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad Collective classification in network data. Al Magazine, 29(3)993, Sep. 2008b. doi: 1009 v29.3.2157. URL http:///oo..aaaaaa.oaaaaaaaaaaaa
aaai org/
article/vieww/2
12

---

Published as a conference paper at ICLR 2026
Akuma Stephen, Tyosar Lubem, and Isaac Adom. Comparing bag of words and tf-idf with different models for hate speech detection from live tweets. International Journal of Information Technology. 14, 09 2022. doi: 110000//411800022000000
Arjun Subramonian, Jian Kang, and Yizhou Sun. Theoretical and empirical insights into the origins of degree bias in graph neural networks. In The eight Annual Conference on Neural Information Processing Systems, 2024. URL
net/ Moorumidd-mmaewwTeee
Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In S. Solla, T. Leen, and K. Müller (eds.), Advances in Neural Information Processing Systems, volume 12. MIT Press, 1999. URL 25507288sbbbcccea00aaaaaaa-----3
neur.ps........p...e........... ccpper /ttpee/ssees//aaer/////////////
Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Yiqi Wang, Jiliang Tang, Charu C. Aggarwal, Prasenjit Mitra, and Suhang Wang. Investigating and mitigating degrec-related biases in graph convolutional networks. Proceedings of the 29th ACM International Conference on Information and Knowledge Management, 2020.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference Learning Representations, 2018.
Haotao Wang, Ziyu Jiang, Yuning You, Yan Han, Gaowen Liu, Jayanth Srinivasa, Ramana Kompella, and Zhangyang " "Atlas" Wang. Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling. In A. Oh, TT. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances Neural Information Processing Systems, volume 36, pp. 50825-50837. Curran Associates, Inc., 2023. URL BRRL/tttsss/pppoceessssssssst nwwrips copppprr files/paper) 7b8 - oone wowferencee.ceee pdf
Yu Wang, Yuying Zhao, Yushun Dong, Huiyuan Chen, Jundong Li, and Tyler Derr. Improving fairness in graph neural networks via mitigating sensitive attribute leakage. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '22 pp. 1938-1944 ACM, August 2022. doi: 10. 10.1145/353467..3555 35339444. http: doi wwt.:/owi.oooooooooooooo 1ttp://....145553551111 3539404
Yuxiang Wang, Xinnan Dai, Wenqi Fan, and Yao Ma. Exploring graph tasks with pure llms: A comprehen- sive benchmark and investigation, 2025.
18771
Bo Wu, Wen-Huang Cheng, Yongdong Zhang, Qiushi Huang, Jintao Li, and Tao Mei. Sequential prediction of social media popularity with deep temporal context networks. arXiv preprint arXiv:17/2.04444; 2017.
Yuxia Wu, Shujie Li, Yuan Fang, and Chuan Shi. Exploring the potential of large language models for heterophilic graphs. arXiv preprint arXiv:2408. 14134, 2024.
Yujun Yan, Milad Hashemi, Kevin Swersky, Yaoqing Yang, and Danai Koutra. Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks. In 2022 IEEE International Conference on Data Mining (ICDM), pp. 1287-1292, 2022.
Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. Network representation learning with rich text information. In IJCAI, pp. 2111-2117, 2015.
Jaewon Yang and Jure Leskovec. Patterns of temporal variation in online media. In Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, WSDM '11, pp. 177-186, New York, NY, USA, 2011. Association for Computing Machinery. ISBN 9781450304931. 10. 10.1145/1935826.1 0E5558226.133886 1935863 URL
doi:
1935863
Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. Learning on large-scale text-attributed graphs via variational inference, 2023. URL http://xxxxvvvxvx.aa/// 2210 14709
13

---

Published as a conference paper at ICLR 2026
Bin Zhou, Xiangyi Meng, and H. Eugene Stanley. Power-law distribution of degree degree distance: better representation of the scale free property of complex networks. Proceedings of the National Academy of Sciences, L17226)14812-148111 2020. doi:
http3//aaa.19890111111 URL https:
//www. //www.pnas..
phws .at.soorrr/oooo
10730 ppa.......... 1 191 8901117
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily graph neural networks: Current limitations and effective designs. In NewrlPS, volume 33. Curran Associates, Inc.. 2020.
Jiong Zhu, Ryan A. Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen K. Ahmed, and Danai Koutra. Graph neural networks with heterophily. Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
14

---

Published as a conference paper at ICLR 2026
A DETAILED RELATED WORK
Static GNN-LLM Fusion. Recent work has focused on two main paradigms for graph learning with
LLMs: LLMs-as-Enhhnnee and LLMs-as-Peedictors (Chen et al. 2024a Li et al. 2024). For LLM-as- Enhancers, LLMS generate enriched node features, ranging from simple text embeddings (Wu et al. 2024) to additional external knowledge (He et al. 2024 Chen et al. (024a), effectively performing automated feature engincering to bolster GNN performance. For LLM-as LLM-as-Predictors. the LLM replaces the GNN as the final classifier, processing serialized graph structure and text as prompts (Wu et al. [024] Wang et al.) 2025). Some variants retain a GNN encoder (Lin et al. 2025). but the LLM still produces the prediction. While both paradigms benefit from the reasoning of LLMs, they also suffer from distinct limitations. Enhancer methods are limited by their static, frozen features while also inheriting the inductive biases of the GNN, retaining the challenges seen with certain structural properties (e.g.. heterophily). In contrast, LLM--s-Preddccto methods can overcome the GNN inductive biases, but may also struggle to encode topology efficiently given prompt length limits and serialization mismatches (Firooz et al. 2025 Wang ct al. 2025). Another common approach to fusing models together is through Mixture of Experts (MoE) which ensemble set of models, each with expert knowledge. While prominent in LIM design (Cai al.
2025) and graph MoE| Wang et al]] (203)), this strategy has yet to be applied to a mixture of L.MMs and
GNNs. Relatedly, LLMs have been shown capable to route different GNNs (Jiang & Luo 2025), yet,
this still limits the pipeline to the inductive biases of the GNNs. Our approach adopts hybric paradigm that selectively invokes the LLM, using expert knowledge from both models, only when it expected to help performance.
Adaptive GNN-LLM Fusion. A challenge for LLM-GNN models is high inference cost, especially when LLMs are used during training. While the static methods above are scalable by treating the LLM as a pre-processer, many methods consider using the LLM within their forward pass, calling the LLM across training and inference (Wu et al. 2024). While attempts to reduce LLM cost via neighborhood sampling or small language model distillation show promise (Fang et al., 2023 Chen et al. [024a), the bottleneck remains where the LLM must be applied to all nodes. Only a few recent works explore adaptive GNN-LLM fusion, where the LLM is invoked selectively rather than universally. E-LLaGNN is one of the first in this space, selecting nodes for LLM augmentation using fixed heuristics based on graph properties such as degree, centrality, or text length (Jiswal et al. 2024). While effective in some settings, these handcrafted rules require manual tuning and can vary in effectiveness across datasets. LOGIN (Qiao al. 2025) introduces a more automated approach, using GNN uncertainty to identify challenging nodes to rewire and simplify message passing. However, this rewiring can disrupt structurally informative but challenging patterns, such as heterophilous links, which can be beneficial in real-world graphs (Luan et al. 2021) ).
LLM-GNN is label-free approach that leverages similar principle as LOGIN, but uses clustering density as proxy of difficulty as opposed to uncertainty (Chen et al. 2024b). Our approach differs from these methods three key ways. First, while different metrics have been proposed route, our systematic study identifies specific structural characteristics that benefit from LLM processing. Additionally, GLANCE does not require prespecified thresholds on these properties, instead learning how to utilize them to route. Second, instead of editing the underlying data to favor GNNs, we preserve its characteristics and invoke the L.M where the GNN struggles. Third, while LOGIN and LLM-GNN address non-differentiabiiiy by editing the underlying data to provide signal from the LLM, retain the true graph signal and directly train the routing policy.
B LARGE LANGUAGE MODEL PROMPTING
In this section, we detail how we generate enhanced features for the base GNN models. Additionally, we provide prompts used to generate LLM embeddings within GLANCE. For both settings, we focus on the Qwen3 family of models, specifically using the Qwen3-Embed--BB model to generate embeddings.
B.1
PROMPTS
We generate node-level text embeddings using (e.g., title and abstract for Cora). We use max prompt length of 1024. Following the model's official configuration, we use last-token pooling and then apply &2 normalization. Concretely, for a batch of tokenized sequences with hidden states H E BRBB Bxdd and (unpadded) last-token indices l. the sentence
Embed-8B by directly processing the node text
15

---

Published as a conference paper at ICLR 2026
embedding for node is
2 = H.,. E 4096
We keep Qwen's include include_promppt behavior enabled, which prepends short instruction prompt internally before the text. The resulting is used as an enhanced feature in place of the shallow features outlined above.
B.2 LOGIN PROMPTS
Following the format introduced in LOGIN, we construct prompts for each node by including the text of the ego node, template that describes the graph and the instructions for the LLM, and the ground-truth labels and GNN predicted labels for nodes in the one-hop and two-hop neighborhood of the ego node.
LOGIN Example for Cora:
NODE <ego-node text>
Given a citation graph
NODE-IDX: <node_id>,
NODE-LIST: [List of all node _id in two-hop neighborhood]
ONE-HCF-MEIGHBORSS [List nede_ id of one-hop neighbors),
INC-HCF-MEIGHECRS [List of node id of two-hop neighbors]
NODE-LAHE1S: [List of ground truth labels]
<GNN-predicted etdd ego node label> where node is the target paper
GNN-PRRO1CTTD--O0D--AAN
and you see the true labels under 'node label'.
Question: Which category does this paper belong to? Choose exactly one from [list of class labels],
Return JSON
{[classification result: <choice>, explanation: <ycur reasoning>)] </END>
B.3 LLM-AS- ALM-ASSPREDICTOR AND GLANCE PROMPTS
When using the LLM predictor, construct prompts for different hops of the neighborhood (ego-only, ego+I-hop, ego+2-hop) and encode them with per-hop information, key design found in heterophilous graph learning, while also helping keep prompt lengths manageable. We utilize the same parameterizations as the L.LM-as-Enhance setting for each of these emheddings, but use prompt length of 512 for the ego node, and 4096 for the one- and two-hop embeddings. Each prompt formatted in a instruction-ques layout compatible with Qwen3 embedding prompts and utilizes last-token pooling of the sequence.
Doing this enables the LLM to retain
The overall strategy in prompt design was to provide the classes and serialized neighborhood structure for node. We do not over-engineer the prompt design, however, this be area for further opportunity in graph learning with LLMs. Additionally, providing additional context on per-dataset basis (e.g. highlighting the domain of the dataset), could provide additional value.
Ego-Only Example:
Instruct: Predict the node's s category from the provided context.
Fossible categories: (list of class labels).
Query:
RGD:
cego-node text>
Category?
</END>
16

---

Published as a conference paper at ICLR 2026
Ego+1 Hop Example:
Instruct: Predict the node's category from the provided context.
Possible categories: [list of class labels].
Query:
EGO:
<ego-node text>
HCP1:
hop neighbor text>
hop neighbor text>
Category?
</END>
Ego+2 Hop Example:
Instruct: Predict the node's category from the provided context.
Possible categories: [list class labels].
Query:
EGO:
<ego-node text>
HOP2:
<2 hop neighbor 1 text>
<2 hop neighbor 2 text>>
Category? </END>
As the degree of the datasets can be large, we use neighbor sampling strategy during prompt construction further maintain manageable prompt sizes. Specifically, we limit the size of each neighborhood be up 5 neighbors per node. Thus, the ego-only text contains one node, the ego + 1-hop contain up to nodes (I + neighbors), and the ego + 2-hop can contain up to nodes (1 + 5*5). use a prompt length of 1024 for the ego-node, and 4096 for the ego I-hop and ego 2-hop. On per-node basis, text capped to a fixed character budget (2000 characters). We process the embeddings using l normalization before concatenation.
When originally fine-tuning the LLM (such as the results in Figure L we apply a 2-layer MLP to map the concatenated embeddings the label space for each model. Then, when incorporating the pre-trained LLM into GLANCE, we remove the MLP head and use the concatenated embeddings directly with the refiner MLP.
C MODEL TRAINING AND HYPERPARAMETERS
We describe here the training protocols for the base GNNs, the I.L.Maas-Prdichhe baseline, and our proposed GLANCE model. Across all models, tune the learning rate from the weight decay from {0.001,0.0 00.001,0.0001,0 w0.0001,00000 0.000001). We apply gradient clipping at [g]|2 S 1.0 for all gradients Training is performed on a single NVIDIA A40 GPU using the AdamW optimizer, with dataset splits of 50/25/25 for train/validation/eess. For DGB-Proodcct we cap the train set to 50,000 nodes for GLANCE, but retain the same test set with the GNN models.
and
C.I GNN (AND MLP) TRAINING
For the GNN backbones, use implementations integrated into PyTorch Geometric when available. Otherwise, for GBK- GNNN and GGCN we use their official implementations. For all GNNs, we use hidden dimension of 64 and batch size of 128. We search Training is run for up to 1000 epochs, with early stopping applied if the validation accuracy does not improve for 30 consecutive epochs. Each model includes one-layer MI.P projection head, which both
network depths of [2.3) layers.
17

---

Published as a conference paper at ICLR 2026
produces the final predictions and allows the learned GNN embeddings to be reused within GLANCE. For heterophilous architectures such as FAGCN, GGCN, and GBK-GNN, we adopt the hyperparameter settings recommended in their official repositories. For the MLP local homophily estimator, we follow similar strategy as the GNNs, using a depth of and hidden dimension 64. Both the GNN and MI.P additionally use ReL.U activations and dropout of 0.2. For OGB- OGB--roddets, patience of 5, and limit the neighbor sampling rate to 30 nodes per neighbor. Additionally, we highlight that the official implementation of GGCN runs out of memory for OGB-Products due to requiring access to the full graph upon initialization. Even with sparse tensors, this is found to be impossible for the DGB-Prooduee dataset.
limit to 20 epochs with
C.2 LOGIN TRAINING
For LOGIN, use GNN backbones with setups described in the previous section. We then estimate the uncertainty of nodes by doing five forward passes with random dropout at a 0.3 dropout rate. We take the top & nodes with defined by:
k=
to query the LLM, where n is the number nodes and 1 h, is the heterophily ratio. While LOGIN originally utilizes the heterophily ratio to query nodes, find this to be too large for Arxiv23 and causes large training times. Thus, the 1000 node cap allows us to utilize LOGIN for Arxiv23. We use a pretrained whether the L.MM predict labels
Vicuna-7B to generate the predictions and explanations. Depending match the ground-truth labels, we either append the explanation to the original node text or prune edges in the node's 's one-hop neighborhood as implemented in LOGIN. We use a similarity threshold of 0.15 for pruning the edges of nodes whose LLM prediction does not match the ground-truth label. Finally, pretrained E5-Large model is used encode the new set of node text that includes the original text and LLM explanations.
C.3
LLM TRAINING
For the LLM baseline, we fine-tune Qwen3-Embed-ae to serve as an LL.M-s-PPrdiichs To reduce both memory and compute, employ parameter-efffiient adaptation via Low-Rank Adaptation (LoRA), inserting adapters into the attention projection layers with a rank of 16 and scaling factor a= 32. We also enable FlashAttention-2 and use 6 precision. During training, only LoRA parameters are updated while the base model remains frozen.
We use a batch size of with gradient accumulation over 8 steps, yielding an effective batch size Training proceeds for a maximum of 10 epochs with early stopping applied if validation accuracy fails to improve for epochs. Since training on all available nodes is infeasible, we cap the training set 3,000 randomly sampled nodes. As with the GNNs, we include a one-layer MLP head to project embeddings into the label space.
C.4 GLANCE TRAINING
When training GLANCE, we first independently train GNN, LLM, and shallow MLP via the training processes explained in Section C and Section C.3
Specifically, each backbone model is trained to optimize the node classification task and is then frozen to ensure its modeling capabilities are maintained within GLANCE. Thus, when training GLANCE, only the router and the refiner models that fuse the two pathways are updated. The router implemented as logistic regression model to predict the likelihood of routing, while the refiner is a 2-layer MLP following the same configuration as the MLP local homophily estimator. Routing decisions are made by selecting the top-K nodes per batch according to the router's predicted scores. To gradually reduce reliance on the LLM during training, we decay the routing budget across epochs = 1,2,.. using an exponential schedule:
K; =o [ennd (eestatt-Knaaar
where Kytart equals the batch size, Kand is the ending routing budget for the last epoch, and is the decay factor. We set Kend = Katart/4 /4 and r r= 0.5 during our experiments, requiring significantly less LLM calls across training.
We train with mixed-preeision using bfloat16, and since no gradients are propagated through the LLM, the memory footprint is greatly reduced, allowing us batch size of 32, For the different datasets,
18

---

Published as a conference paper at ICLR 2026
set gun equal to the batch size. As in the LLM baseline, cap the training set at 3,000 nodes. Early stopping is applied with a patience 2 epochs. We additionally tune the LLM query cost B (0.1,0.2.0.3). and set the router weight Aronter 1.0 and entropy regularization weight Aent =0.01 by default.
D DATASET DETAILS
To characterize the limitations of current heuristics route, well as evaluate GLANCE, we utilize three widely studied TAG benchmarks: Cora (Mecallum et al.] 2000). Pubmed (Sen et al. (000a)) and utilize OGB-Products (Hu al. 2020).
Arxiv23 (He et al. 2024). Then, for our study on scalability, representing a multiple order of magnitude increase in dataset size compared to typically studied TAGs.
These datasets also differ significandly in homophily levels, allowing to probe routing decisions under diverse conditions.
. Cora. Cora is citation network consisting of machine learning papers. Papers share an edge if they share citation. The data is from http:::/tt/ttt. htt://gittuii com/my www/omolassaaaraaeaaaaaoaaaaaaaa sasbarry
LLM- bew.hharkingg (Wang et al., (2055). The canonical feature vector for each paper is a bag-of-words feature vector. The nodes are categorized into different Al subfields. Cora possesses the highest global homophily level out of the datasets (h.80.
Pubmed. Pubmed is larger citation graph on biomedical articles, where papers share an edge if they share a citation. The data is ftomtttps: : /tt:://gtittt wwt.c/ommmmm... com/myflaahibarry LLM www.hmarccakk........... iing Each article is represented by TF-IDF word vector, with labels corresponding to different disease types. Similar to Cora, Pubmed exhibits high global homophily (f==0.0m 0.80), but also posseses a larger pocket of heterophilous nodes (see Figure I
Arxiv23. Arxiv23 is a recently curated citation subgraph of arXiv covering computer science papers published after 2023, surpassing the knowledge cutoff of flagship models like ChatGPT- 3.5. The data is from /ttps://gits thub. com/ Xi aox nHe/ tape__ arxiv_ v0223 We find that the original proposed Arxiv23 dataset has a large number of isolated nodes, which can confound the evaluation metrics given over half of the nodes do not possess graph structure. Thus, we subset Arxiv23 to only include the largest connected component. Each article represented by Word2vee embedding, with each class corresponding Cora and Pubmed, Arxiv23 displays a wider array of homophilous and heterophilous edges, with 0..70.67), making a challenging benchmark for neighborhoodd-averaginn GNNs.
computer science subfield. Unlike
Arxiv-Year Arxiv Year is heterophilous variant of -Axvv, using the publication year the label. The data is fronfhttpps://obb..ttaafff tile ranges (see ht https:
edul but with the labels created via quan-
cwm CUAT CNoCUAAAiNNNnAAA NowN//ooooooohhhho Homophi ly- Sca le/ py producing 5 labels. The dataset is roughly an order of magnitude larger compared to the three previous datasets. For this dataset, we only use enhanced features from the raw text. Arxiv-ycar is heterophilous 0.22).
bllob/master//atteoo
OGB-Products OGB-Products is a large-scale e-commerce TAG provided from the OGB hittps://ogb... sww.afrr...............mo.. edul The task is to predict the category of e-commerce items, where items share link if co-purchased. The dataset is roughly two orders of magnitude larger than the TAGs seen in previous literature, with 47 classes. We only use enhanced features for this dataset. OGB-Products is homophilous 00.8)
The explicit number of features, classes, and other key metrics are summarized in the table below. Notably, the datasets we study contain wide variety of sizes, densities, and homophily levels.
Table 6: Summary statistics of the datasets used in our experiments.
Dataset
#Nodes
#Edges
Original Feature Dim
#Classes
Global
Cora
2,708
5,278
1433
0.81
Pubmed
19,717
44,324
500
3
0.80
Arxiv23
19,550
55,350
300
40
0.67
Arxiv-Year
169,343
1,166,243
5
0.22
OGB-Products
2,449,029
61,859,140
47
0.81
19

---

Published as a conference paper at ICLR 2026
Table 7: NCS for difficulty, d..,, and uncertainty- based routing is compared against random baseline which randomly routes nodes to LLM. Higher NCS better. Numbers parenthesis denotes number nodes routed.
Cora
Pubmed
Arxiv23
10%
15%
20%
10%
15%
20%
10%
15%
20%
Routing Strat.
(68)
(102)
(136)
(193)
(740)
(986)
(189)
(734)
(978)
Random
-0.029
-0.01
-0.029
0.049
0.062
0.062
0.145
0.083
0.092
C-density
-0.044
-0.069
-0.074
0.065
0.07
0.065
0.096
0.091
0.092
GCN
Degree (d.)
-0.029
-0.02
-0.037
0.059
0.059
0.064
0.084
0.094
0.105
Uncertainty
-0.147
-0.088
-0.044
0.183
0.15
0.141
0.139
0.144
0.14
Random
-0.044
-0.069
-0.029
0.041
0.028
0.053
0.078
0.038
0.058
C-density
-0.015
-0.049
-0.044
0.041
0.045
0.045
0.08
0.079
0.077
GCNII
Degree (d.)
-0.029
-0.02
-0.029
0.059
0.055
0.053
0.063
0.068
0.078
Uncertainty
-0.147
-0.088
-0.051
0.132
0.107
0.117
0.117
0.134
0.127
Table 8: NCS for estimated and true homophily routing strategies with different routing top-k percentages. Higher NCS is better. Numbers in parenthesis denotes number of nodes routed. Homophily is shown to produce high NCS values, with both the estimated and true variants producing larger NCS values le increases.
Cora
Pubmed
Arxiv23
15%
20%
10%
15%
20%
10%
15%
20%
Routing Strat.
(68)
(102)
(136)
(493)
(740)
(986)
(489)
(734)
(978)
b 0.265
0.118
0.081
0.314
0.299
0.266
0.217
0.217
0.212
GCN
hw -0.059
-0.078
-0.074
0.215
0.199
0.193
0.196
0.192
0.187
dy -0.059
-0.069
-0.051
0.055
0.041
0.038
0.02
0.059
0.057
be 0.235
0.098
0.066
0,213
0.2
0.168
0.176
0.159
0.15
GCNII
feg -0.074
-0.069
-0.074
0.154
0.131
0.123
0.137
0.114
0.122
d, 0.0
-0.039
-0.037
0.045
0.026
0.022
0.033
0.064
0.056
E ROUTING FOR GCN AND GCNII
In addition to our enhanced results provided the main section of the paper, we include supplemental results with routing the base GCN and GCNII architectures with their original features.
E.1 ROUTING WITH PRIOR METHODS
Similar to the results seen on the enhanced model variants, we see that in Table across Cora, Pubmed, and Arxiv23, uncertainty-baxe routing consistently delivers the strongest NCS among the heuristic baselines for both backbones, with degree and C-density lagging behind (often near or below the random baseline on Cora). This result confirms our original finding that, when routing into an LLM, predictive uncertainty is the most reliable heuristic from the prior methods, while the other signals alone can even harm performance.
E.2 HOMOPHILY
BAOSE BASED ROUTING (ESTIMATED VS. TRUE)
We begin by evaluating the effectiveness of the true local homophily, h,. We can see in Table similar to the main text, h, achieves the highest NCS across models and datasets. However, when we move to estimated homophily h., we can see that it follows similar relative trend. Relative degree d, maintains its relatively weaker performance. These findings further confirm that homophily is powerful routing signal, with hn conferring robust gains.
20

---

Published as a conference paper at ICLR 2026
Table 9: Correlation between model uncertainty and (true) homophily.
Dataset
Corr(uncertainty, Ai)
Corr(uncertainty. h2)
Cora
-0,68
-0.334
Pubmed
-0.883
-0.331
Arxiv23
-0,407
-0.381
GCN (Shallow)
GCN (Entanced)
GCNI (Shallow)
GCNI (Enhanced)
7+
Cora
Pubmed
OMN Accuaccy
Duggos
74
Arxi23
035
1.0
035
6.6
6.00
0.25
Homoppily
Figure 4: Stratified Performance Based on Homophily and Degree. Darker red denotes instances where the LLM performs best, and darker blue denotes instances where the GNN performs best. Across methods, we see further deviation in performance compared the individual metrics
E.3 CORRELATION BETWEEN HOMOPIILY AND UNCERTAINTY
Table shows a moderate correlation (inverse) between model uncertainty and local homophily across all datasets. Intuitively, as homophily decreases (i.e., more label conflict neighborhoods), uncertainty increases. This empirically links the two most effective routing cues where regions of low homophily tend be high-uneertainty. However, this correlation is relatively weak, thus we hypothesize including both metrics can provide complimentary information, i.c. the signals are not providing redundant information.
E.4 STRATIFIED ANALYSIS ON PUBMED
In Figure we observe consistent trends with those reported on Cora and Arxiv23. Specifically, while performance across models converges the high homophily regime, LLM models deliver pronounced gains in the low-homophily and low-degree settings. outperforming strong GNN backbones by substantial margins. For example, LLM-based predictions exceed the next best GNN-enhanced variant by as much as 10.4% on heterophilous nodes (h, < 0.20). These findings further confirm that the advantage of LLM augmentation is not dataset-specific, generalizes across datasets, highlighting homophily as a reliable signal for routing.
E.5 HOMOPHILY AND DEGREE INTERPLAY
While homophily and degree display diverging trends between LLM and GNN training, highlighting their complimentary strengths, it also known that their interplay can further inform GNN behavior. In Figure 4 stratify across both properties for each model and dataset, further demonstrating increased divergence in
21

---

Published as a conference paper at ICLR 2026
K-d
K-d6
Palmed
Arav223
302
Losal
Local
Figure 6: Stratified Performance for Different K at Test-Time We use a batch size of 32 when testing GLANCE, and use different routing budgets depending K. We find that performance tends to increase for heterophilous groups of nodes as we increase K, demonstrating GLANCE's ability to take advantage larger routing budgets.
performance differences. For instance, while Pubmed attains 10% difference when looking at homophily isolation, this difference can go high as ~30% under the same setting (e.g., g., for GCNII (enhanced) on Cora, low homophily and low degree can experience a significant improvement compared to high homophily and high degree).
ADDITIONAL ABLATION AND SENSITIVITY ANALYSES
this section, we present supplemental analyses intro- duced in Section 6.3 We begin by examining the sensi- tivity of GLANCE to the parameter K, which controls the number of nodes routed to the LLM at test time. This analysis reveals how performance changes as the bud-
Pubmed
Accuracy (Homophlyy Stratified)
14000
0.9
12000
0.8
get for LLM queries is adjusted, offering insight into
10000
the trade-off between predictive accuracy and computa- tional cost. Understanding this trade-off is particularly important in practice, as practitioners may wish to tune K align with real-world latency or cost constraints. Then, we ablate the routing features used by GLANCE to determine which signals are most critical for effective routing. By systematically removing each feature, we isolate their individual contributions to routing accuracy and downstream performance. Together, these supple- mental analyses deepen our understanding of GLANCE's behavior, confirming both its robustness different query budgets and its reliance on principled routing features that target GNN failure regimes.
0.6
8000
GCN [Sid.)
6000
Node
0.4
GCNI (Std.)
4000
2000
0.2
LLM
0.38-0.20
0.200.400
040 sd
0.8-88300 80
4.86 0g
Accuracy (Relative Degree Stratified)
Local Homophily
3.00
0.95
0.90
9000
0.86
6000
0.80
Node
0.75
2000
F.1 PERCENT CHANGE DURING ROUTING
0.65
0.60
0.0-0.25
023.0,30
0.73-1.0 n0
1.00-20.00 on
As introduced in Section 6.3 we analyze the effect of varying the routing budget (8,12,16) GLANCE's performance. repoorss stratified results across dif- ferent levels of local homophily. allowing us to assess how additional LLM queries impact different subpopulations of nodes. We observe that increasing K generally im- proves performance on heterophilous and low-homophily nodes, where the GNN struggles most. Importantly, with = 16, GLANCE consistently achieves the highest overall performance across routing strategies, demon- strating that allocating larger query budget enables the framework to more effectively target the nodes where LLM assistance is most beneficial. This highlights a practical trade-off where larger K increases computational cost but can yield substantial accuracy improvements, particularly in challenging structural regimes.
0.30
Relative Degree
Figure 5: Stratified Performance. Performance given local homophily (top) relative degree (bottom): bars denote property distributions (right y-axis).
22

---

Published as a conference paper at ICLR 2026
Table 10: Stratified and Overall Performance for Standard Routing 12) and Full Routing (K'=22). A denotes difference from 12 to =32 Green indicates improvement, red indicates a decrease. While routing can improve heterophilous nodes further, this can come at the cost of performance on homophilous nodes.
Dataset
0.00-0.25
0.25-0.50
0.50-0.75
0.75-1.00
Overall
K = 12 K =2
46,44:2.4
54,41:1.6
79.7+-0.9
98.1+:0.2
89.5
Cora
49.2:1:.44
68.94:3.2
76.9+:2.1
96.94-0.88
88.3+11.7
+2.8
+14.5
-2.8
-1.2
-1.2
K 12 K = A
71.5++0.5
65.9++0.4
89.4+:0.4
97.7++0.1
92.6+-0.1
PubMed
82,4+-077
79.24:..3
91.1++0.9
98.0++0099
94.1+-0.2
+10.9
+13.3
+1.7
+0.3
+1.5
K 12 K = A
45.2++0.7
63 63,24:0.2
85.9+0.3
95.3++0.0
82. 82.1
Arxiv23
19.7
65.8+112
82.5+
94.0++0.2
81.6+0.4
+4.5
+2.6
-3,4
-1.3
-0.5
Table 11: Routed Accuracy Breakdown: GLANCE vs. GCNIL We K = 12 with batch size of 32. We compare the accuracy on the subset nodes routed by GLANCE to the LLM, denoted R. against the baseline where the GCNII backbone makes predictions for the exact same set R. The final row highlights the performance improvement gained by routing these nodes to an LLM via GLANCE. The large positive improvements demonstrate that GLANCE effectively identifies and routes difficult nodes that benefit significantly from LLM inference.
Model on Routed Set R
Cora
Pubmed
Arxiv23
GLANCE (Routed to LLM)
87.6++0.9
91.6:+0.5
77.1+00.8
GCNII (Routed to GCNII)
82.7+=166
85.2+00.9
72.6+2.4
Improvement (GLANCE vs. GCNII)
+4,9
+6.4
+4.5
Full Routing, To further illustrate the value of selective routing, evaluate the setting where every node in a batch is routed to the LLM (denoted as K = for a batch size of 32), allowing the LLM to influence all predictions. As shown in Table IO full routing is not necessarily beneficial. reinforcing the need for GLANCE's selective strategy. Specifically, while a higher routing budget improves performance on the most challenging nodes (those the lowest homophily bins) we also observe a decline in accuracy on high homophily nodes. Consequently, both Cora and Arxiv23 see reduced overall accuracy despite the substantially higher query cost. This trend highlights an important trade-off: although LLMs can correct difficult nodes, indiscriminately routing all nodes dilutes accuracy where the GNN already performs well and significantly increases computational cost. In contrast, GLANCE's selective routing (e.g.. = 12) delivers substantial gains heterophilous nodes while preserving strong performance on high homophily regions, all with far fewer LLM calls. We also note that while Pubmed does not exhibit as pronounced of drop, this result expected Pubmed's node text has been shown to be highly informative, enabling LLMs to predict node labels accurately in zero-shot settings (Chen et al., (0024)). However, despite this capability, the performance gap between K = 12 and K = 32 remains extremely small in the hompphlyy bin, indicating that the significant increase in computation offers minimal benefit.
F.2 GRANULAR ANALYSIS ON ROUTING PERFORMANCE
In this section, provide deeper analysis on the performance differences between routed and non-routed nodes. Moreover, we offer additional studies to characterize performance under a random router.
F.2.1 PERFORMANCE IMPROVEMENTS FROM LLM ROUTING
To characterize the effectiveness of GLANCE's routing, evaluate the predictive accuracy specifically on the subset of nodes routed to the LLM, denoted R. We compare the performance of GLANCE against a baseline where predictions for this exact same set R made solely by the GCNII backbone. This direct comparison provides clear evidence for whether invoking the LLM improves prediction accuracy for the nodes selected by our router. The results shown in Table I1
Findings. We compare the accuracies on the routed nodes R using predictions from the GCNI backbone versus the I.I.M Across all datasets, routing these selected nodes the L.LM yields substantially higher
23

---

Published as a conference paper at ICLR 2026
Table 12: Comparison of Standard Random Routing for GLANCE. Random routing selects nodes randomly from each batch for LLM refinement, while Standard uses the router leamed within GLANCE. The A accuracy rows report the difference in accuracy for each method across overall, routed, and non-routed accuracy. Positive As (green) indicate improvement with learned routing over random routing. Overall, GLANCE's learned router consistently improves performance, particularly on the non-routed subsets where standard GNNs excel.
Routing Strategy
Metric
Cora
Pubmed
Arxiv23
Overall Acc
86.4+1.11
91.4410.88
80.3++0.9
Random
Routed R. Accuracy
83.3++2.3
91.9-1177
74.8++0.9
Non-routed R Accuracy
87.5++022
91.2++0.3
82.1+0.8
Overall Acc
89.5++0.4
92.6++0.1
82.1+0.1
Standard (as used in GLANCE)
Routed R. Accuracy
87.6+0.99
91.6++0.5
77.1+0.8
Non-routed R' Accuracy
9..
92.9+066
84.0+0.7
Overall
+3.1
+1.2
+1.8
Accuracy (Standard Random)
Routed R
+4.3
-0.3
+2.3
Non-routed
+2.7
+1.7
+1.9
accuracy. Specifically, GLANCE improves performance on R by +4.9% (Cora), +6.4% (PubMed), and +4.5% (Arxiv23), These significant gains demonstrate that GLANCE's routing policy is highly effective at identifying challenging nodes that directly benefit from the enhanced capabilities of the LLM.
F.2.2 RANDOM ROUTING PERFORMANCE
To contextualize the routing improvements further, evaluate a random routing baseline that, for the same query budget (K= 12), selects nodes at random while keeping all other components of GLANCE unchanged. This experiment serves natural lower bound routing quality and serves as another indicator to whether GLANCE is able to identify difficult nodes. We measure the difference overall, routed, and non-routed performance between the standard and random routing schemes in Table 12 where more positive is better. We perform 3 runs per experiment and report standard deviations.
Findings. We fisst compare GLANCE's learned router (denoted Standard) against random routing baseline. Under random routing, overall performance degrades across all datasets, with accuracy dropping by up 3.1% (Cora). More importantly accuracy on the routed subset is generally lower than with the learned router, suffering drops of 4.3% and 2.3% on Cora and Arxiv23, respectively. This result demonstrates that randomly selected nodes benefit less from LLM refinement. While PubMed exhibits a marginal 0.3% increase on R. under random routing, the ron-routed accuracy R' drops consistently across all datasets. This indicates that a random policy frequently wastes L.M queries on simpler nodes that the base GNN could already classify correctly, leaving behind challenging nodes that the GNN struggles to resolve. Together, these results reinforce that GLANCC''s lightweight routing reliably identifies semantically and structurally difficult nodes, yielding targeted performance gains where standard GNNs fail.
Anxv23
1.0-
1.-
0.5
0.5
0.0
-05
-15
AIC
Figure 7: Ablation Study Over Routing Features Overall Performance. Each plot denotes the performance changes, relative to full GLANCE performance, when training without one of the routing features. Performance typically docays across datasets and features, highlighting the benefit of each feature GLANCE's robust performance.
24

---

Published as a conference paper at ICLR 2026
05
02
00
Alcurayy (h)
00-
70
72
020-022
0.8.0.
2 2.906.775
0.854 80
a 8.0.625
8.6-0.75
075-1120
0.8
223-6.0
076677
875-100
Figure 8: Ablation Study Over Routing Features Stratified Performance. Each of bars denotes the stratified performance GLANCE when training without one the routing features. The gray bar denotes full GLANCE training. We find that the largest performance drops occur in the heterophilous regions, highlighting that GLANCE specifically targeting these difficult nodes.
Table 13: Per-bin accuracy across homophily levels for GLANCE trained with different & strength Larger 3 requires routed nodes have more substantial impact. We find that increasing 3 tends to systematically increase performance on heterophilous nodes.
Cora
Pubmed
Arxiv23
3 0.00-0225 0.25-0.50 0.50-0.75 0.75-1.00 0.00-0.25
0.25-0.50 0.50-0.75 0.75-1.00 0.00-0.25
0.25-00.50
0.50-0,75 0.75-1.00
0.0
53.3
97.8
67.2
97.5
42.4
60.9
86.8
95.6
GLANCE
46.4
54.
79.7
98.1
70.1
68.7
89.2
97.6
45.2
63.2
86.0
95.3
0.2
47.1
53.3
80.0
97.8
70.6
65.7
88.6
97.7
45.2
62.9
85.2
95.5
0.3
51.0
53.3
79.0
97.8
71.3
65.7
89.2
97.8
45.5
63.2
85.1
95.6
F.3 ROUTING FEATURE SENSITIVITY
While each routing feature is motivated either by prior work analysis Section we quantify their contribution through an ablation study. For each dataset, we retrain GLANCE while removing each feature turn and measure the resulting change in accuracy. Figure reports the performance drop relative the full model, showing that overall accuracy consistently declines when any feature is removed. Notably, the impact varies across datasets where no single feature dominates universally, underscoring that different structural signals matter in different settings. This trend is further evident in Figure g where, within stratified bins, removing a single feature produces substantial degradations in performance.
E.4 DIFFERENT ROUTING PENALTIES
In this section, we provide additional sensitivity analysis on 8, GLANCE's penalty term. Increasing B, the margin a routed node must surpass for the LLM to be selected, systematically re-allocates L.LM usage toward the hardest, most heterophilous nodes. As shown Table I3 raising 3 improves accuracy in the powest-homophiy bin across Cora (from 45.1 to 51.0), Pubmed (from 68.0 to 71.3), and Arxiv23 (from 42.4 to 45.5), while leaving the most homophilous bin effectively unchanged and producing only small drops in the intermediate bins. In practice, the router spends fewer calls marginal gains in the intermediary homophily levels where signal is weaker and concentrates budget where the GNN can be most improved upon. This behavior makes / simple, yet effective, knob trade-off homophilous and heterophilous nodes. When leveraging GLANCE, we recommend tuning 3 on validation metrics in the range (0. (0.3- determine the most effective value.
G SCALING GLANCE TO LARGER DATASETS
Having evaluated GLANCE's performance on small- to medium-scale TAG benchmarks, we now turn to scalability. A concemn with LLM-GNN fusion is whether additional components introduced by method introduce significant overheads, especially when moving to much larger datasets. To address this, we first break down the runtime of GLANCE into its major components: (i) GNN computation, (ii) LLM computation, and (ii) -LANCf-speciffic modules, namely the router, feature generation, and refinement stages. As shown in Figure the results reveal a clear trend where the I.LM dominates the overall runtime,
25

---

Published as a conference paper at ICLR 2026
the BLAANCE-speciiiii components extremely lightweight, constituting only negligible fraction of total cost. Both routing and refinement involve simple feed-forward operations over low-dimensional features, while feature generation is largely pre-computed within the pipeline. This finding has two key implications. First, it validates our design choice to offload expensive reasoning exclusively to the I.I,, while keeping routing and refinement efficient. Second, it demonstrates that scaling GLANCE larger graphs is primarily limited by the LLM's cost, rather than by the fusion mechanism itself. We therefore this motivation apply GLANCE the substantially larger dataset OGB-Products in Table showcasing its ability to scale while retaining efficiency and accuracy.
GNN
Route Pestures Refine
LLM
13
03910
1.-
23020
03
72
16
0
32
72
1.
Tha-c Reuted (SS-33)
Figure 9: Runtime breakdown of GLANCE. LLM computation dominates overall runtime, while GLANCT-speccfiic modules only negligible overhead, confirming the framework's scalability to larger datasets.
Computational Cost. We analyze the computational complexity of GLANCE by isolating the costs of its three key components: (1) the routing features, (2) the router, and (3) the refiner. As GLANCE operates on top of frozen GNN and L.M encoders and is agnostic their internal architectures, specific GNN or LLM cost. Instead, we focus on the GLANCE-specine
do not define elements. For each component, we denote the cost of forward and backward pass, when applicable, by T Tww and 2ww respectively.
Notation. Let denote the batch size, k the routing budget per batch, p the dimensionality of the routing feature vector, e the number of classes for dataset, and dmax the maximum degree of any node the batch. We also define the following per-node costs:
CaNN: cost of applying the frozen GNN encoder to one node,
CLIM: cost of an LLM forward pass for one routed node,
Co: cost of applying the homophily MLP to one node,
Cet cost applying the refiner MLP to one routed node.
Building Routing Features (Per Batch), Routing features consist of: (1) GNN embeddings, (2) GNN uncertainty, (3) degree information, (4) original node features, and (5) homophily estimates. Using the above per-node costs, this results the per-batch cost:
T fwdd feat
(BCoNN) +
where R the number of stochastic GNN passes used for uncertainty.
Router:
The router applies linear transformation the features and selects the top-k nodes to route:
T Touter O(Bp) Bl)
T Treeer = O(Bp).
Selective LLM Queries. GLANCE queries the LLM for the routed nodes (with no backward pass):
dwd = O(KCLMM)
Refiner MIP. For each routed node, GLANCE applies the refiner MLP:
Swd = O(KCret
bad = O(CCrer)
Overall Per-Batch Complexity. Combining all components yields:
huich
( (LL+ tmax bogg + +N(CCuL+
GNN & Routing Feature
LLM and Refiner
26

---

Published as a conference paper at ICLR 2026
Interpretation. GLANCE's complexity is comprised of two terms: (1) the full batch processing that creates routing features leveraging the GNN, and (2) the LLM and refiner computation applied only to the k routed nodes. Given CLLM far exceeds the other terms (as shown in Figure D. one can scale up the GNN, router, or refiner without incurring large relative increases in runtime. Moreover, because training uses eiiie--rrdient--ttlee optimization procedure, the cost differential between training and inference is small as the backward pass is only on the cheaper components of the pipeline.
27


Reviews:
[{"review_id": "L9hWfCXHYt", "reviewer": "Reviewer_JFZW", "summary": "The paper argues that current GNN-LLM hybrids waste LLM calls by applying a single fusion rule to all nodes, which hides where LLMs actually help. By stratifying nodes with respect to local homophily and relative degree, the authors show that GNNs win in high-homophily/high-degree regions, while LLMs win in heterophilous/low-degree pockets. They then introduce GLANCE learns a lightweight per-node router to route top-k nodes to the LLM and fuses GNN+LLM via a small refiner head. On four datasets, GLANCE shows the improvement and it does so with a small query budget.", "questions": "1. In Section 5.1.1 it is not fully specified how the GNN backbone F is obtained before it is used inside GLANCE. Could the authors explain how it is pre-trained?\n\nI would be happy to raise the score if the above concerns can be addressed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The studied problem of deciding when to invoke an LLM for graph node prediction is timely; the observation that LLM benefits are uneven across nodes is clearly argued and well grounded in the analysis.\n2. The proposed GLANCE framework is coherent with routing, context construction, and fusion stages reinforcing one another; learning to gate LLM usage from cheap structure aware signals is a sensible and practical design.\n3. The reported gains on heterophilous and low degree subsets, together with consistent improvements in overall accuracy under a limited query budget, indicate that selective LLM supervision actually strengthens node level recognition.", "weaknesses": "1. Since GLANCE routes only a subset of nodes to the LLM for cost reasons, it would be useful to report the result which sends every node through the LLM GNN fusion path. Even if this is expensive, such a result could act as an empirical upper bound for the proposed router and show how close the learned routing actually gets to the best case. It would also be helpful to understand the trade-off between budget cost and model performance.\n2. Why did the authors include heterophilous GNNs when the datasets in Table 2 are all homophilous, where these models are unlikely to be advantageous? It would be more informative to include genuinely heterophilous or non citation graphs, since the current experiments are mostly on citation networks; this would make the empirical evaluation of the proposed method more convincing.\n3. To better understand the quality of the routing decision, it would help to break down accuracy by routed vs non routed nodes. Among the nodes that the router decided to send to the LLM, how many became correct due to LLM input, and how many stayed wrong? Among the nodes that were not routed, how many were already correct under GNN only? This breakdown would show whether the router is actually focusing LLM budget on the hard regions of the graph.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3mZAdQpavF", "reviewer": "Reviewer_qBh6", "summary": "This paper addresses the inefficiency and uneven performance of uniform GNN-LLM fusion for text-attributed graphs and proposes GLANCE, a node-aware fusion framework. It identifies that GNNs excel at high-local-homophily/high-degree nodes while LLMs outperform on heterophilous/low-degree nodes, with local homophily as a signal for LLM benefit. The proposed approach GLANCE freezes pre-trained GNN and LLM, trains a lightweight router to invoke LLMs on hard nodes selectively, and uses a refiner MLP to fuse embeddings.", "questions": "In addition to the questions raised above, I have two more questions to be answered:\n\n- Does the paper employ random sampling rather than structure-aware sampling (such as prioritizing neighbors with high degrees or edge weights)? Could this simplification restrict the LLM’s capacity to capture fine-grained structural correlations?\n- Is an adaptive $\\beta$ that adjusts based on real-time computational constraints better than a fixed one? Would this affect the tradeoff between heterophilous node accuracy and inference speed?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper proposes a node-aware GNN-LLM fusion mechanism, targeting the inefficiency of uniform LLM invocation and filling a gap in TAG learning.\n- The proposed GLANCE takes local homophily as an LLM-benefit signal, combined with a homophily estimation scheme, balancing practical deployment applicability.\n- GLANCE achieves good performance across different graph datasets, while freezing pre-trained models ensures scalability for large-scale TAGs like OGB-Products.", "weaknesses": "- The label-free homophily estimate relies on a trained MLP to predict node labels. This is rather counterintuitive: since this is essentially a node classification task, why is an MLP capable of fulfilling this role instead of a GNN? Furthermore, additional justification is required regarding how this MLP achieves robust performance. For instance, on low-resource data (with few training labels) or high-noise data (with short/noisy node text). If the MLP’s label predictions are inaccurate, will the estimated homophily become a misleading routing signal?\n- Given the complex nature of graphs, uncertainty, local homophily, and degree are not necessarily consistent in their implications for label prediction, as also observed in Appendix E.3. Nodes with high homophily/degree may not exhibit low uncertainty (as well as high prediction success); conversely, nodes with low homophily/degree might show high GNN certainty. The manner in which the router interprets these conflicting signals remains an issue of interpretability.\n- To control prompt length, GLANCE samples only 5 neighbors per hop for LLM input. This may result in the discard of critical structural information (e.g., a key neighbor of a heterophilous node that explains its label being excluded from sampling).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RrZts9eECa", "reviewer": "Reviewer_8RQH", "summary": "The paper proposes GLANCE, a selective GNN–LLM fusion for text-attributed graphs. A router decides which nodes “deserve” LLM embeddings; others use the base GNN only. The routing signal mixes estimated homophily, uncertainty, degree, and representation cues. A refiner fuses GNN and LLM features when routed. Training uses a cost-aware counterfactual reward that prefers routes which reduce supervised loss under a budget. Experiments on several TAG benchmarks report stronger accuracy on low-homophily regions while keeping LLM calls sparse. Ablations suggest homophily features drive much of the routing benefit, and sensitivity studies show larger budgets help heterophilous nodes most.", "questions": "1) **Batching for top-k?**  \nHow do you form batches—random nodes or topology-aware sampling?\n\n2) **Neighborhood sampling details?**  \nFor ego/1-hop/2-hop text: what sampler, sizes, randomness, seeds, and truncation/token caps?\n\n3) **Refiner capacity and overfitting?**  \nSince GNN/LLM are frozen, what is the refiner’s depth/width/parameter count vs. the GNN head? Any overfitting on small graphs (e.g., Cora)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) New method.\nA simple, modular selective GNN–LLM fusion (router + refiner on frozen backbones) that learns when to invoke LLM embeddings under a budget. Clear cost–accuracy tradeoff and easy to plug into existing TAG pipelines.\n\n2) Extensive experiments.\nBroad evaluations across multiple text-attributed graph benchmarks, with routing rates, ablations, and sensitivity to budgets. Results consistently show gains where graphs are hard (low homophily) while keeping LLM calls sparse.\n\n3) Interesting task.\nAddresses a timely, practical problem: cost-aware reasoning on text-attributed graphs. Selectively augmenting nodes with LLM features makes TAG learning more applicable to real deployments with tight compute limits.", "weaknesses": "**Weakness**\n\n1) **Router vs. eval is entangled.**  \nYou use homophily to route and also to judge gains. That can create circularity. Show results when the router does **not** use homophily (or uses only degree/uncertainty). Also slice by other structures (roles, motifs, k-core, betweenness). The ablation drop for \\(h_v<0.5\\) strengthens this concern.\n\n2) **Homophily estimator lacks detail.**  \nYou build \\(\\hat h_v\\) with an auxiliary classifier \\(Q\\). Specify \\(Q\\)’s architecture and size. Report loss, early stopping, and regularization. Clarify train/val label usage and leakage risk. Add calibration metrics (ECE/Brier).\n\n3) **Missing bounds and fuller ablations.**  \nAdd an **oracle router** (LLM correct, GNN wrong) as an upper bound. Add a **shuffled router** with the same budget as a lower bound. This will contextualize NCS and show headroom. Also run **only-X** and **pairwise** feature ablations for redundancy/synergy. The big drop when removing homophily for \\(h_v<0.5\\) is not the whole story.\n\n\n**Suggestions**\n\n1) **Counterfactual reward may be biased.**  \nIt uses labels and batch top-k, and only for routed items. That can skew learning. Consider off-policy estimates, stochastic routing, or periodic global calibration.\n\n2) **Test LLM and prompt diversity.**  \nYou rely on one LLM/embedding and one multi-level prompt. Try other sizes/models and one-level vs. multi-level prompts. This separates routing effects from encoder/prompt artifacts.\n\n3) **Disentangle routing vs. feature gains.**  \nAdd a control that routes “hard” nodes but **does not** call the LLM. Reweight the GNN head only. If gains persist, they may come from attention, not LLM content.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zice8sK0ue", "reviewer": "Reviewer_r4iU", "summary": "This paper studies leveraging LLMs to assist with the node classification task and introduces a novel approach: selectively invoking LLMs for specific nodes. Through an empirical study, the authors identify homophily as a key factor in determining whether predictions should rely on LLMs or GNNs. Building on this insight, the paper proposes the GLANCE framework, which integrates a router component for selecting nodes to be processed by LLMs, a fusion component to combine embeddings from GNNs and LLMs, and a reward-based strategy to optimize both components stably. Extensive experiments on citation networks demonstrate GLANCE's improvements for low-homophily nodes and its scalability to larger e-commerce networks. \n\nThis work introduces a promising direction for graph learning, i.e., **selectively invoking LLMs only when necessary** to balance performance and efficiency.", "questions": "1. For Weakness 1, is my understanding of the training pipeline correct? Additionally, could the authors define the notations used in Figure 1?\n\n2. Have the authors considered removing the router loss entirely and jointly optimizing the router and refiner solely based on task prediction loss? If so, how does this alternative approach compare in terms of performance and stability?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **This paper is well-motivated and explores a novel research direction**. It is intuitive that GNNs and LLMs are complementary: GNNs excel in high-homophily scenarios, while LLMs can significantly improve predictions for heterophilic graphs (as supported by prior works like [Ref A]). However, no prior work has investigated the selective usage of LLMs to improve performance for heterophilic nodes while maintaining computational efficiency.\n\n2. **The preliminary experiments are comprehensive and provide meaningful insights**. To determine which nodes should be processed by LLMs instead of GNNs, the authors first analyze structural features like degree and clustering density, then identify homophily as the decisive factor. Finally, they propose a label-free metric, estimated local homophily, to guide the routing. This multi-step analysis provides a clear and logical progression for deriving the proposed factor.\n\n3. **The GLANCE framework is carefully designed to align with the problem's motivation and is grounded in rational principles**. During inference, GLANCE achieves efficiency by using only two feature forward passes: one for the router to decide whether to invoke LLMs and another for combining complementary embeddings. During training, the authors incorporate both task-level prediction loss and a policy-gradient-inspired loss to optimize the routing policy, ensuring robust performance.\n\n4. **The experimental evaluation is extensive and promising**, demonstrating GLANCE's improvement in predictive performance, its controllable computational cost, and its scalability to large-scale graphs. The results convincingly show that GLANCE is effective, particularly for low-homophily nodes.\n\n--- \n\n[Ref A] Wu, Xixi et al. *When Do LLMs Help With Node Classification? A Comprehensive Analysis*. In ICML, 2025", "weaknesses": "1. While the paper is generally well-written and easy to follow, some minor points require clarification. For example:\n     * In Figure 1, the notations, e.g., GCN (Std.), GCN (Enh), LLM, are unclear. What do they represent?\n     * In Section 5, the authors state that GNNs and LLMs are pre-trained, but the training pipeline is ambiguous. My understanding is that GNNs and LLMs are first trained separately on each dataset using classification loss, and then these two encoders are frozen during the GLANCE framework's joint optimization of the router and feature refiner. If this understanding is incorrect, clarifying the exact training process would be helpful.\n\n2. While the authors provide a general efficiency study to showcase GLANCE’s computational cost, a more detailed analysis would enhance the paper, e.g., what is the computational complexity of GLANCE during inference, and could the authors provide a breakdown of costs for both training and inference?\n\n3. Although the experiments cover datasets of varying scales, they are still limited in diversity. All four datasets exhibit relatively high homophily, meaning heterophilic nodes form only a small portion. As a result, the overall performance improvements from GLANCE are marginal. Extending the experiments to more heterophilic datasets, e.g., the original arXiv dataset, would better validate GLANCE’s effectiveness for heterophilic scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)