GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedback
Authors: Ruiyao Xu
Kaize Ding
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Graph Neural Networks
Graph Few-shot Learning
Semi-Supervised Learning
Primary area: learning on graphs and other geometries & topologies
TL;DR: We propose GNN-as-Judge, a framework that leverages GNNs' feedback to select reliable pseudo-labels and a weakly supervised fine-tuning approach for tuning LLMs.

Abstract: Large Language Models (LLMs) have shown strong performance on text-attributed graphs (TAGs) due to their superior semantic understanding ability on textual node features. However, their effectiveness as predictors in the low-resource setting, where labeled nodes are severely limited and scarce, remains constrained since fine-tuning LLMs usually requires sufficient labeled data, especially when the TAG shows complex structural patterns. In essence, this paper targets two key challenges: (i) the difficulty of generating and selecting reliable pseudo labels on TAGs for LLMs, and (ii) the need to mitigate potential label noise when fine-tuning LLMs with pseudo labels. To counter the challenges, we propose a new framework, GNN-as-Judge, which can unleash the power of LLMs for few-shot semi-supervised learning on TAGs by incorporating the structural inductive bias of Graph Neural Networks (GNNs). Specifically, GNN-as-Judge introduces a collaborative pseudo-labeling strategy that first identifies the most influenced unlabeled nodes from labeled nodes, then exploits both the agreement and disagreement patterns between LLMs and GNNs to generate reliable labels. Furthermore, we develop a weakly-supervised LLM fine-tuning algorithm that can distill the knowledge from informative pseudo labels while mitigating the potential label noise. Experiments on multiple TAG datasets demonstrate that GNN-as-Judge significantly outperforms existing methods, particularly in low-resource regimes where labeled data are scarce.

Published as a conference paper at ICLR 2026
GNN-AS- UNLEASHING THE POWER OF LLMs FOR GRAPH LEARNING WITH GNN FEEDBACK
Ruiyao Xu, Kaize Ding
Department of Statistics and Data Science Northwestern University
rui ruiyaoxu2028Qu.co hww.:r/oeeeeeettotttttteeeeeee nowtwwesterrt. edu ka i ze wiwgghoowwwtteeeernnn edu
ABSTRACT
Large Language Models (LLMs) have shown strong performance text-attributed graphs (TAGs) due to their superior semantic understanding ability on textual node features. However, their effectiveness as predictors in the low-resource setting, where labeled nodes are severely limited and scarce, remains constrained since fine-tuning LLMs usually requires sufficient labeled data, especially when the TAG shows complex structural patterns. In essence, this paper targets two key challenges: (i) the difficulty of generating and selecting reliable pseudo labels TAGs for LLMs, and (ii) the need mitigate potential label noise when fine- tuning LLMs with pseudo labels. To counter the challenges, we propose a new framework, GNN-as-Judae
which can unleash the power of LLMs for few-shot semi-supervised learning on TAGs by incorporating the structural inductive bias of Graph Neural Networks (GNNs). Specifically, GNN-as-Judge introduces collabo- rative pseudo-labeling strategy that first identifies the most influenced unlabeled nodes from labeled nodes, then exploits both the agreement and disagreement patterns between LLMs and GNNs to generate reliable labels. Furthermore, we develop weakly-supervised LLM fine-tuning algorithm that can distill the knowl- edge from informative pseudo labels while mitigating the potential label noise. Experiments on multiple TAG datasets demonstrate that GNN-as- GNN-ax-Judge signifi- cantly outperforms existing methods, particularly in low-resource regimes where labeled data are scarce.
INTRODUCTION
Text-Attributed Graphs (TAGs), where nodes correspond text documents and edges represent their relationships, are pervasive across various applications such as citation networks, social media platforms, and e-commerce ecosystems (Chang & Blei, 2009; Yang et al., 2015; Hu et al., 2020; Wu et al., 2025). Unlike conventional attributed graphs, TAGs encode raw textual contents rather than numerical values, which requires more dedicated mechanisms to effectively capture semantic information while preserving structural relationships. Recent advances in Large Language Models (LLMs) have shown exceptional capabilities in text understanding (Brown et al., 2020; Dong et al., 2022; Minace et al., 2024), driving growing interest in leveraging their exceptional text understanding capabilities to address TAG-related tasks (Chen al., 2024b; Tang et al., 2024; Ye et al., 2024; Chen et al., 2024c; Hu et al., 2024; Wang et al., 2025; Wu et al., 2025). Many previous studies investigate LLMM-as-Prediche approaches, which employ LLMS as direct predictors by integrating graph context through graph encoders or carefully crafted prompts (Tang et al., 2024; Chen et al., 2024a;b).
It noteworthy that current research of LLMs-as -reeiictrssssssss for node classification on TAGs, primarily focuses on the supervised setting where abundant labeled data is accessible. This is mainly because LLMS lack GNNs message passing mechanisms to leverage unlabeled nodes, and thus require sufficient supervision signals for effective fine-tuning (Chen et al., 2024a; Tang et al., 2024; Ye et al., 2024). However, real-world graphs are usually sparsely labeled, and thus directly applying existing methods to such few-shot semi- supervisd setting (Ding et al., 2020; Wan et al., 2021; Ding et al.,
Code available https: //g www.:tthbcccoococe htt.uu..com/uuo0000 /NNN- as- -uduee..

---

Published as a conference paper at ICLR 2026
2022; Yu et al., 2025) will easily lead to overfitting and poor generalization (Wu et al., 2025; Tang et al., 2024; Ye et al., 2024). Although one can leverage pseudo-labeling techniques to augment the limited labeled training data (Lee et al., 2013; Rizve et al., 2021; Qiao et al., 2018; Liu et al., 2022; Sun et al., 2020; Li et al., 2018), previous studies have recognized that "easy" pseudo lahels with high confidence provide limited learning signal, whereas "hard" samples are more informative but introduce greater label noise (Bengio et al., 2009; Kumar et al., 2010; Mukherjee & Awadallah, 2020). This challenge becomes even more pronounced when incorporating LLMs, leading to the following two unresolved challenges in few-shot semi- supervised node classification on TAGs:
How to go beyond the knowledge of LLMs to obtain reliable pseudo-labeled data? Since LLMs are inherently difficult to interpret complex graph structural patterns (Huang et al., 2024; Guo et al., 2023), solely relying on LLMs with no structural inductive bias to generate pseudo labels is less desirable. More critically, not all unlabeled nodes are equally valuable for pseudo-labeling thus, selecting the most influential subset of unlabeled nodes crucial for maximizing performance under computational constraints. Although rext-based serialization methods try to encode structural context into the prompts, those methods could still struggle with self- self-genratee pscudo labels due the hallucination and self-bias of L.LMs (Chen et al., 2024b; Luo et al., 2024; Liu al., 2022; Gao et al., 2024). Generating reliable pseudo-labeled examples that encode not only textual but also structural inductive biases is therefore crucial for enabling LLMS to transcend their inherent knowledge limitations on graphs.
e How to extract the knowledge from pseudo-labeled data while miligating the potential label noise during LLM fine-tuning tineetuning Despite pscudo-labeling showing its empirical effectiveness in many fields (Lee et al., 2013; Rizve et al., 2021), the potential label noise has been longstanding problem. Especially for those "hard" pseudo-labeled examples that are more valuable for training the model, they could also bring more label noise if the labels are incorrect. Simply performing LLM supervised fine-tuning with the noisy pseudo labels can lead to performance degradation (Shumailov al., 2023; Kim al., 2023; Cheng et al., 2020), which necessitates the need to develop new learning algorithm that can effectively distill the knowledge and mitigate the label noise when fine-tuning with pseudo-labeled data.
In this paper, we propose GNN-as-hdge, a novel framework that fine-tunes LLMs on sparsely labeled graphs using feedback from GNNs. Instead of mining the "easy" and "hard" pseudo-labeled nodes solely based on LLM itself like standard self-training approaches (Ma et al., 2017; Lee et al., 2013; Mukherjee Awadallah, 2020), at its core, a GNN with structural inductive bias acts as a judge to provide additional guidance for LLM to generate reliable pseudo-labels. GNN-as-Judge strategically leverages both agreement and disagreement between GNN and LLM as signals to identify not only "easy", but more remarkably, "hard" pseudo labels that LLMs are more likely to make mistakes. To further mitigate the potential label noise, especially in the harder examples, we develop
LLM fine-tuning algorithm that jointly performs fine-tuning on the two selected pseudo-labeled node sets. Specifically, in addition to applying supervised LLM instruction tuning on the agreement (easy) node set, we propose to conduct preference tuning on the disagreement (hard) node set, which allows LLM to learn relative preferences between predictions from the two models. To summarize, our contributions are mainly three-folds:
We study the problem of fundamental but underexplored research problem, where the key challenges lie in selecting reliable pseudo labels and mitigating label noise during fine-tuning.
for graph few-shot semi remi-supervised learning,
We propose GNN-as-Judge
novel framework that positions GNNs as judges to select reliable pseudo labels for fine-tuning LLMs. GNN-as-Judge is also equipeed with a new fine-tuning algorithm that can further mitigate label noise during LLM fine-tuning.
We conduct comprehensive experiments on different TAG datasets with various scales. Results demonstrate that GNN-as-Judg significantly outperforms both traditional GNN-based approaches and other Lased baselines, especially in extreme low-resource scenarios.
2 RELATED WORK
LLMs for Graph Learning. Recent research has extensively explored applying (L)LMs to TAGs, yielding significant advancements in feature encoding, node classification, and link prediction (Chen et al., 2024b; Tang et al., 2024; Ye et al., 2024; Hu et al., 2024; Wang et al., 2025; Wu et al., 2025). Early work primarily employed small-scale pretrained language models such BERT (Devlin et al.,
2

---

Published as a conference paper at ICLR 2026
2019) or RoBERTa (Liu al., 2019) text feature encoders to extract enhanced representations for graph learning ( (Yang et al., 2021; Li et al., 2023; Wen & Fang, 2023; Zhao et al., 2023). With the advent of powerful large language models such ChatGPT (Achiam et al., 2023), researchers began utilizing these models in two primary ways: as-enhancers and as-predictors.
methods (He al., 2024; Chen et al., 2024e; Sheng al., 2025; Yu et al., 2025; Liu et al., 2024; Wang et al., 2025) utilizes LLMS to generate enhanced knowledge in the form of explanations, embeddings or labels based on the original graph data. For example, TAPE (He et al., 2024) uses --gnerreeteed explanations as enriched node features for GNN learning. MaMM-----redictorr methods frames graph problems as text generation tasks, directly utilizing LLMs as predictors to output classification or prediction results by transforming graph structures into natural language descriptions (Wang et al., 2024a; Ye et al.,, 2024; Chen et al., 2024a; Tang et al., 2024; Wang et al., 2024b; Chen et al., 2024b; Hu et al., 2024). For instance, L.LGA (Chen et al., 2024a) introduces template-har based graph-to-text conversion with a specialized projector for structure comprehension, while GraphGPT (Tang et al., 2024) performs multi-stage instruction tuning to align LLMs with graph patterns. Although LLM-as- Predictors methods show promising performance and zero-shot transfer ability, they typically assume abundant labeled nodes are available. Recent work (Wu al., 2025) demonstrates that standalone LLMs are weak predictors with limited labeled data, and incorporating graph structure is essential for achieving satisfactory performance.
Pseudo Label Selection. In semi-supervised learning, pseudo-labeling (Lee et al., 2013; Shi et al., 2018) serves as an effective solution by augmenting limited labeled datasets with generated labels. To mitigate the potential label noise, previous research usually leverages model's confidence to select "casy" samples that are considered to be clean (Ma et al., 2017; Kumar et al., 2010). Recent research, however, argues that there is little information to gain with these "easy" examples (Mukherjee & Awadallah, 2020) and merely relying on high- high-confidence examples may make the model self-biased (Rizve et al., 2021; ; Liu et al., 2022). Consequently, current research emphasizes the importance of mining both "easy" and "hard" samples during training to maximize model performance (Liu et al., 2022; Mukherjee & Awadallah, 2020; Shrivastava et al., 2016). Nevertheless, identifying the "casiness" or "hardness" " of samples is often non non-trivial .specilll for LLMs.
PROPOSED APPROACH
In this section, we propose an LLM-GNN pseudo co-labeling framework GNN-as-Judge that addresses LLM pscudo-labeling challenges through three core designs as shown in Figure 1: (i) a subset selection strategy that identifies nodes with most information from labeled nodes for pseudo labeling; (ii) collaborative pseudo label selection mechanism that leverages GNN as complementary signal sources for LLM to generate high-quality pseudo labels, and (iii) a weakly-supervisee fine-tuning algorithm that mitigates label noise when fine-tuning LLM with generated pseudo labels.
3.1 PRELIMINARIES
We consider the problem of few-shot semi-supervised node classification on text-attributed graphs (TAGs) defined as G (V,E, A, X), where V = {11, D2...... uNy the node set, & S V V the edge set, A E {0, 1 is the adjacency matrix, and X E TR N contains node features derived from text. Given a small labeled set Verai where each class c E (l,.... C} has exactly & labeled nodes, along with validation set V.at, the goal is to predict labels for test nodes Viest V (Miiin Uvai). Common approaches include: (1)
methods that learn node representations via message passing, yielding predictions g.NN fa(u, A, X) with trainable parameters Ø, and (2) LLM-as-Predicees Predictors methods that construct prompts P(vs) from textual attributes and generate predictions GLLL (v.)( using language models with parameters A.
3.2 GNN-AS- FOR LLM PSEUDO LABELING ON GRAPHI
To break the bottleneck of using ALM-generated pseudo labels, our approach tries to mine "easy" and "hard" pseudo labeled nodes based on the agreement and disagreement between LLM and GNN, in order to provide reliable pseudo labels for fine-tuning the LLMs. To avoid the computational constraints of pseudo-labeling over the entire unlabeled set, we first identify the most influential unlabeled nodes according graph structures, then apply our collaborative pseudo labeling framework
3

---

Published as a conference paper at ICLR 2026
Input Text
Prediction
Agrevment Set
Tite connineuugi fiteue
PUO
43.4
hhat Intertee And te sn Alyurtan tites in leret of dratetsu arghes being ore the stmedit...
Tuming
Ext
Iellccee-Cadbed
Sdeeted Nodes
PUO
Perado Label Setectian
Input Graph
GNN Prediction
Preference Yuningg
Lpr
THUNNN Gald URERE Dusss
Disagromentt
Figure I: Framework of GNN-as-uudge for few-shot semi-supervised node classification on TAGs.
to this selected subset. Specifically, we leverage the complementary strengths of two distinct models:
a structure-aware GNN fe and text-centric LLM Ma. both trained on the labeled set Virain
Celtuence-Guided Node Selection for Pseudo Labeling. Due to computational constraints and efficiency considerations, is crucial to select a candidate node subset from the entire unlabeled node set for pseudo labeling. While LLMs excel at processing textual information, they lack awareness of graph structures and cannot readily capture the influence of labeled data on unlabeled nodes (Dai et al., 2025; Chen et al., 2024b). To address this, we employ GNNs as structural proxy identify subsets of nodes that carry the most information from labeled data. We leverage the concept of node inflence that quantifies the extent which one labeled node's 's representation can impact another unlabeled node's 's representation through the graph structure (Xu et al., 2018; Huang & Zitnik, 2020; Wang & Leskovec, 2020; Zhang al., 2021; Wang et al., 2022). We first provide the formal definition:
Definition (Node Influence). Node influence Ini,s of node is on node vg in the final GNN output [xx()))
(cs) is: Ing., = -|| /0x!.. (o0 where (00) and Xx, are the final node representations of v; and vj learned by the neighborhood aggregation mechanism after infinite layers, the norm is any subordinate norm, and the Jacobian measures how a change in the representation of 0 translates to a change in the representation of U-j-
Definition captures how effectively labeled nodes can propagate their information to unlabeled nodes. Nodes with high influence from labeled data are ideal candidates for pseudo labeling because: (i) they receive stronger signals from the labeled set, making pseudo labels more reliable, and (ii) they better reflect the distributional characteristics of the labeled nodes, ensuring representativeness and diversity. We then propose Theorem 1, which establishes that the influence of node i on node fy decays with their distance and provides computable upper bound for node influence. The proof is provided in Appendix D.I.
Theorem 1.
Let Puray denote the set of all paths between nodes v; and Vj and let Piccey Pvvce denote the set of shortest paths. For any path t Purxy. let D' f be the geometric mean of node degrees occurring on path t defined D'M (IIrret Drcn,)) 1/1s] where [t] is the path length and Duae v. is the degree of node Uk. Define Dam mintePi, (DoM) as the minimum geometric mean among shortest paths, ht d(v,, Uz) as the shortest path distance between vs and Vy. and (P as the number of shortest paths between vs and Vy. Then, the node influence Loy,e, from vs to Uj satisfies:
P P
Tr = 1|oxx000 /0x(00)xxx(
S
(D'GM)'
Based on Theorem we define the influence score of an unlabeled node By E Vunlabeled as its maximum influence from any labeled node:
PP' P (D )4-
IS(ny) = max To = max
(2)
V.VV
4

---

Published as a conference paper at ICLR 2026
We then rank all unlabeled nodes based on their influence scores and select the top K nodes with the highest values:
Velleced = TopK ((IS(1)) : by E Vunlabeled K))
(3)
This formulation ensures we select unlabeled nodes that are most strongly influenced by the labeled
node set, maximizing the potential for effective information propagation during pseudo labeling. After obtaining the predictions from both the GNN and LLM on the selected node set Vselecsed we further partition the selected unlabeled node set into agreement node set V Vagreed {u, E Vielected o, " and disagreement node set Vdisagreed { Vellcced ú, + ū!LM).
3. The intuition
behind using both agreement and disagreement node sets is that they serve complementary roles in effective LLM fine-tuning. While agreement nodes provide more reliable learning signals that consolidate the model's 's understanding, carefully selected disagreement nodes offer challenging examples that provide more informative learning signals.
Agreement Node Set Selection with GNN Feedback. We assume that nodes in the agreement set are more likely to have correct pseudo labels, as agreement between models with distinct inductive biases suggests higher label reliability. We provide theoretical justification for this assumption, which shows that the expected accuracy of the agreement set is strictly than that of either individual model:
Theorem 2. Consider Vielected with ground truth labels y" Let PLLM and PONN denote the individual accuracies of the LLM and GNN respectively. Under conditional independence of model errors due to different inductive biases and uniform error distribution across incorrect classes, the accuracy of the agreement set satisfies:
P(u (e E Vugreed) IV
PILM PGNN
1I-PLM)( (1 Doaw
PLIM PGNN
C-1
where C is the number of classes. Furthermore, this lower bound exceeds max(puM,, PGXN) when both models perform better than random guessing.
The proof is provided in Appendix D.2. This theoretical result confirms that the agreement set provides higher-quality pseudo labels compared to using either model individually.
Disagreement Node Set Selection with GNN Feedback. While pseudo- labeled nodes in the agreement node set are provably more reliable, simply using easy, self-generated labels where the LLM already predicts correctly provides limited new learning signals and may lead to overfitting. To counter this issue, we propose to employ GNN as a judge for selecting high-quality pseudo labels in the disagreement node set. Since our method selects structurally influential nodes for pseudo labeling and the GNN's message-passing mechanism can exploit local neighborhood information that inaccessible to the LLM,, the GNN is assumed to be more reliable than L.MM for pseudo labeling in this set. To further ensure the quality of pseudo labels, we utilize GNN's probability distribution over different classes as a natural indicator of its preference strength. For each node 0, E Vdisagreed. we compute preference score measuring how strongly the GNN favors its own prediction over the LLM's prediction: (p,)((;)) = PuNN(U8 (u. CNNN v.) - PaNx( IIM
vz),, where PUNN GNN (e,) is the GNN's predicted probability for its top class, and PGNN(G) C v.)) is the probability assigned to the
LLM's predicted class. A larger preference score indicates a stronger conviction by the GNN in its
own prediction relative to the LLM's alternative. We then select the final subset of nodes, V Vissgrredd- by retaining only those where the GNN's preference score exceeds predefined confidence threshold Vdisugreed Sper(vs) 2 T).
V disagreed
[U;-
Directly utilizing LLMs to evaluate "easiness" or "hardness" of unlabeled nodes on TAGs is non-trivial. Our strategy utilize the GNN as a pseudo label judge to effectively identify "easy" and "hard" samples, providing practical method to identify reliable pseudo labels.
Remark:
3.3 LLM WEAKLY-SUPERN SUPERVISED FINE-TUNING ON GRAPHS WITH GNN FEEDBACK
Sunkly-Supervieee Fine-tuning Algorithm. Based on our selected pseudo-labeled nodes, we tne-tuning algorithm that fine-tuna munet LLMs graphs using unified objective. Our approach integrates both instruction tuning and preference tuning into a single training framework:
propose
[())- ((e...) (TT((: 2e, v.)] AI Ww. ) [LT Mi, Ym (..)
(5)
5

---

Published as a conference paper at ICLR 2026
where D Dagreed represents the data distribution over the agreement node set Vagreed. Diisagreed represents the data distribution over the selected disagreement node set V disagreed and À controls the contribution of the preference tuning loss. For each selected node ets we construct the input 2; as the node's textual content formatted with task-specific prompts (detailed prompt templates are provided in Appendix E). For nodes Vagreed, y/ represents the consensus prediction where both GNN and LLM agree. For nodes in D Dissggred'' + Yur; and yt,, represent the GNN's prediction (preferred) and LLM's prediction (dispreferred) respectively.
LLM Instruction Tuning with LLM-GNN Agreement. For nodes in the agreement set V Vagreed apply instruction tuning (Ouyang et al., 2022) to reinforce correct predictions. Given an input node text feature 22 and the agreed pseudo label yi, the instruction tuning loss is defined as:
Cr v2 V/) log po (o(ys (24),
(6)
where (o (z;) represents the L.M's probability of generating the label y/ given the node text Zi. and denotes the model parameters.
LLM Preference Tuning with LLM-GNN Disagreement. Directly fine-tuning the LLM on pseudo-labels from the disagreement set via standard instruction tuning is problematic because the disagreement node set contains potentially more label noise compared to the agreement node set. To address this, we reframe the problem on the disagreement set using a preference tuning objective. For each node U; E V disagread we construct a preference pair where the GNN's prediction yie, serves as the preferred response and the LLM's prediction Ht, serves as the dispreferred response. This formulation enables the model to learn from the relative relationship between competing outputs without requiring absolute correctness from either prediction. The preference tuning objective is formulated as:
(7) where 22 represents the input prompt for node V/ Iw.s and yu are the preferred and dispreferred outputs respectively, and g-(-) is a preference function that scores the relative preference between the two outputs. In our implementation, we adopt Odds Ratio Preference Optimization (ORPO) (Hong
CPT (0; 24, Sw 31,s) - log or (go Fw.kr B(,e))
et al., 2024), which uses the log odds ratio as the preference function:
oddsa x))
go Bur VI) = logg odds (3I x)
(8)
where oddso(y x:) Pa(y x1/(1 Pa x)). By minimizing this loss over V disagreed the
LLM learns to increase the relative likelihood of the GNN's predictions compared to its own initial predictions. This approach mitigates the risk of overfitting to noisy lseudo-labels while still leveraging the valuable disagreement signal to improve model performance.
Remark: Our framework can be considered as LLM preference alignment framework by replacing human feedback with signals derived from GNNs. While we implement preference tuning using ORPO (Hong et al., 2024), our approach is also compatible with other methods like such as DPO (Rafailov et al., 2023), SimPO (Meng et al., 2024), and other variants.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. We train and evaluate our framework on four widely-used benchmark datasets for few- shot semi-supervised node classification: Cora (Yang et al., 2016), Citeseer (Sen et al., 2008), Pubmed (Sen et al., 2008), ogbn-arxis v and ogbn-products (Namata et al., 2012). For Cora, Citeseer and P Pubmed, which are three most widely used citation networks, we follow the experimental setup of previous work Gasteiger et al. (2018) and split each dataset into training (i.e., K nodes per class for K-shot task), validation set, and test set. To evaluate performance on large-scale graphs, we also include ogbn-arxiv and ogbn-productt Benchmark (OGB) (Hu et al., 2020). Detailed statistics of all datasets summarized in Appendix A.
from the Open Graph
Baselines and Implementation Details. To evaluate the effectiveness of our proposed framework. compare it against baseline methods from three primary categories: (1) Classical GNN Models:

---

Published as a conference paper at ICLR 2026
We include established graph neural network models such GCN (Kipf & Welling, 2017) and SGC (Wu et al., 2019); (2) CaMM-as-rredictoo We also include baselines that utilize general- purpose LLMs with various prompting strategies such as zero-shot, chain-of-thought, and neighbor- augmented prompting (Wei et al., 2022; Chen et al., 2024b); (3) I.IM-Graph Methods: Aligning with foeus, benchmark against various state-of-the-ar LLM-Graph methods, including GLEM (Zhao et al., 2023), TAPE (He et al., 2024), LLM-GNN (Chen et al., 2024c), LLaGA (Chen al., 2024a) and GraphGPT (Tang al., 2024). We use GCN (Kipf & Welling, 2017) and Llama-3- 8B- SB-Instruct (Grattafiori et al., 2024) as backbone models of our approach. Further descriptions and implementation details on all methods are provided the Appendix B and C.
4.2 FEW-SHOT SEMI-SUPERVISED NODE CLASSIFICATION
Table I: Node classification accuracy (%) across different datasets and shot settings. Results show mean + standard deviation performance. Best results are bolded, while second-best results are underlined.
Shol
Method
Cora
Citeseer
Pubmed
ogbn-arxiv
oghn-products
GCN
69.45:2..34
63.12+1.00
65.23:t:.67
38.33-1:2.41
59.19:0079
67,21
63.07=00.55
65.34:1.118
39.16+1.49
57.42=1.12
7ero-Shot
65.54.1..26
18.17:10.64
74.51
50.18:11.44
75.48 1.54
Graph-CoT
63.02=0.77
47.23=11.00
2.242
49.67::1.23
74.15= 1.83
3-hhot
w. Neighbor
18.72.11.55
54.9311 1.28
74.98113.16
49.28:1:2.09
1.07 76.88
GLEM
67.81
$3.09 1:
63.85:1:002
36.37:1:1.96
$2.4611.99 1,93
TAPE
73.71
04.96.10.36
71.33:1.0.87
48.25:10.77
69.64 1.15
6..61
4..36-1:.772
LLaGA
54.79.10.91
12,93 1.24
43.96.1:1.3
29.73.12282
30.67 1.54
GraphiPTT
57.77=1.62
52.34
17.51
31.26:22273
20.8342.97
GNN-as-Judge
1.289
73.59+00644
87.12:+-0.89
62.21=1145
81.02 1,23
GCN
13.69:11.0
63,24:10.87
70.58:00.49
45.67:10.41
68.26:11.54
SGC
72.48:10.35
62.08
70.74.10.94
49.89 1..23 1.23
66.78 1,56
Zero Shot
65,54:00.26
$8.17:20.64
74.51::0.39
50.18:2:.44
75.48
5-shot
Gruph CoT
63.02.10.77
1..23:1.1.6
86.22.12.47
49.67 1.23
74.15
Neighbor
68.7241.56
$4.93
34.98:33.16
29,28:22.09
76.88
GLEM
74.69-10.46
61.71+00.58
73,29
1.55
56.87 1.70
TAPE
74.28=0081
67.73+00.48
03.02-+-0.83
$5.22-+0:48
1.34
LLM LLM-GNN
1..6111100
62.37:11.35
74.35-10.95
45.74:1 1.66
64.01 0.339
LLaGA
62.88-22.19
4..71
18.6
33.74::2.445
37 3.12
GraphGPT
60.17::1.44
51.83-1:2.24
$7.39:13.67
36.25:11.88
44.78:1:2.34
GNN-as-Judge
GCN
78.22-10.89
68.38:11 1.49
75.33:1:094
50.95-11.77
09.65-10.89
SGC
78.49=0.37
67.44:00600
1.998
1.23
67.91
68.54-10.26
75.48
10-shot
Graph-CoT
63.02=0.77
47,23++1.00
26,22+22447
49.67
74.15=1.83
w. Neighbor
68.72:11.56
54.93+11.28
74.98:3..16
49, 49,28:+22.09
76.88 1.07
GLEM
78.11:20.73
66833+0.61
74.17:42.39
47.73-+1.00
60.22 1.89
79.33:10.57
69.39-10.65
17.1
60.37-10..2
0.65
LLM-GNN
79.39
66.28400.44
76,82+00.57
52.7420048
66 00.39
LLLGA
69.25
51.22-L1.43
6..29.1...6
45.35
40.55 1.63
GraphiPTT
61.58-10.77
55.40-1..16
71.33:4-2.8
48.67-11.89
$1.46 1.05
GNN-as-Judge
80.71:20.83
74.62++1.35
90.17:+1.69
67.88:11003
82.48 1,56
Table presents the comparative performance of approach against various baseline methods across multiple datasets and experimental settings. Our results demonstrate three key observations. Observation 1: Our proposed GNN-as-Judhe Juddge consistently outperforms all baseline methods across all datasets and experimental settings, demonstrating superior robustness. Observation 2: Our method shows particularly strong performance in low-resource settings, highlighting its effectiveness in real-world applications. In extreme scenarios such as 3-shot and 5-shot settings, GNN-as-Jude maintains substantial performance advantages, making it highly practical for domains where labeled data is scarce or expensive to obtain. Observation 3: In low-resource settings, GNN-based methods consistently show strong performance compared to LLM--bsed approaches. Traditional GNN architectures effectively exploit graph structure and connectivity patterns even with limited labeled data, while L.MM-aased methods struggle due to insufficient textual context and unreliable pseudo label generation in these constrained scenarios.
7

---

Published as a conference paper at ICLR 2026
4.3 POOSS-DAASSET ZERO-SHE SHROT NODE CLASSIFICATION
In this section, we evaluate the zero-shot generalization capabilities of GNN-as-Judge. We compared with various Prreiictrrr graph learning models on the ogbn-arxil dataset and evaluated their zero-shot performance on Cora, Ci Citeseer, and Pubmed without additional fine-tuning. Unlike traditional GNNs that require task-specific classifica- tion heads.
Table 2: Zero-Shot Cross-Dataset Node Classification.
methods can perform zero-shot learning across different label sets. As shown in Table 2, GNN- as-Judge demonstrates superior zero-shot transfer performance across all datasets. substantially outperforms GraphGPT and LLaGA, which struggle with cross-dataset generalization. Their approach of encoding graph structure into tokens appears to constrain the LLM's generalization capabilities, resulting in per- formance worse than untuned base LLMs. These results indicate that GNN-as-Judge is more robust to distribution shifts between graph datasets and better preserves the LLM's inherent gener- alization capabilities while incorporating graph-based insights. This makes our approach particularly valuable for practical ap- plications where labeled data may be scarce or available for domains, necessitating models that can effectively generalize to new, unseen graph structures.
Test
Accuracy
ogbn-arxiv
LLAGA
16.24:10.55
GraphGPT
6.29=0.73
GNN-as-Judge
68.27:100.11
oghn-arxiv
LLAGA
14.72.11.12
GraphGPT
5.37=00.44
Citeseer
GNN-as-Judge
56.67:20..9
ogbn-arxiv
LLAGA
30.52:::1.18
GraphGPT
10.54:1..005
Pubmed
GNN-as-Judge
83.41:10.76
4.4 ANALYSIS ON PSEUDO LABEL SELECTION
To evaluate the effectiveness of our influence-guided node selection strategy, we conduct a comprehensive analysis comparing different pseudo label selection approaches. Figure 2 illus- trates the accuracy of 1,500 pseudo labels selected using various methods across three repre- sentative datasets under 3-shot setting and then annotated by GCN (Kipf & Welling, 2017). We apply the same GNN-as-Judge filtering process after ini- tial node selection. We analyze GNN-as-Judge approach alongside several alternative selection strategies: (1) Ran- dom: Randomly selecting pseudo labels; (2) Degree (Page et al., 1999): Selecting nodes based on their degree central- ity in the graph; (3) AGE (Cai et al., 2017): Using graph embedding to identify representative nodes; and (4) LLM- GNN (Chen et al., 2024c): Specifically, we use the proposed difficulty-aware strategy combined with AGE (Cai et al., 2017). As shown in Figure 2, our influence-guide tion consistently outperforms all baseline approaches across datasets, achieving the highest pscudo label accuracy after applying the same filtering process. The results highlight that structural influence from labeled nodes provides a more effective criterion for pseudo label selection compared to simple graph topology measures. More detailed quantitative analysis of pseudo label selection is provided in Appendix F.
Pscado Label Accuracy (S)
85
60-
75
70-
Degres
AGE
LAMU-GNN
60
55
Cora
Citeseer
orbi-arxiv
Dataset
Figure 2: Comparison of pseudo label selection strategies across datasets.
selec-
4.5 ABLATION STUDY
In this section, we conduct an ablation study to analyze the contribution of key modules from our proposed GNN- Judgee
als Meudly sxpirtied
CWI eutorrr
Cora
El Closser
Pohmed
ogbn-arsiv
onln-preducts
723
Figure 3: Ablation study demonstrating the contribution of each component in our framework.
Effectiveness of Pseudo Label. We first examine the contribution of pseudo labels by evaluating the w/o Pseudo Labels variant that only performs supervised fine-tuning on the labeled set without incorporating pseudo-labeled nodes. As shown in Figure 3, removing pseudo labels leads to
8

---

Published as a conference paper at ICLR 2026
significant performance degradation across all datasets. This demonstrates that eseudo-labelling strategy effectively expands the training set with high-quality pseudo labels.
Effectiveness of Disagreement Node Set. We then evaluate the importance of the disagreement node set through the w/o Disagreement Node Ser variant by excluding these challenging examples from our training data. Removing the disagreement node set leads to significant performance degradation. These nodes, identified through prediction disagreements between different model variants, represent the most informative hard examples that can provide additional learning signals,
Effectiveness of Weakly-Supervised Fine-Tuning Tuning, Finally, we assess the contribution of our proposed weakly supervised fine-tuning strategy through the w/o Weakly supervised iite-tuning variant by replacing it with standard instruction tuning. Our full GNN-as-Julge approach consis- tently outperforms standard instruction tuning on selected nodes with particularly notable gains on Pubmed due to potentially more label noise in the disagreement set. Unlike standard instruction tuning on pseudo-labeled data, our approach is better equipped to handle the inherent uncertainty potential noise present in the pseudo labels of the selected nodes.
4.6 SENSITIVITY AND TRAINING TIME ANALYSIS
In this section, we evaluate the framework's 's sensitivity to different computational efficiency. Additional experiments and analysis provided in Appendix G.
and analyze the
Sensitivity Analysis. We investigate the impact of two critical hyperparameters in GNN-as-Judge: the top-K selection for unlabeled nodes in pseudo-labeling and the preference score threshold for disagreement node filtering. This analysis is essential for understanding the robustness and the scalability of our approach.
For the top-K analysis, we vary the number of selected unlabeled nodes within the range K E (500, 1000, 1500, 2000, 2500]. Fig- ure 4 demonstrates that GNN-as- Judge exhibits scalability properties, with performance generally improv- ing as more unlabeled nodes are in- corporated into the pseudo-labeling process and then used for training. However, the performance increase is moderate as more data points are incorporated, suggesting diminishing returns beyond a certain threshold while maintaining computational efficiency.
Acourary (he
Caru
75
76
76
500
1900
2000
2000
0.5
0.7
0.
Dop &
Preference Score Threshold
Figure 4: Sensitivity analysis of byperparameters: top-K unlabeled nodes (left) and preference score threshold (right) across three benchmark datasets.
For the preference score threshold т, we examine values in the range E {0.1, 0.5, 0.7, 0.9) understand how preference score filtering affects model performance. Notably, the perfor- mance variations are relatively small when ranges from 0.5 to 0.9. This robustness is particularly valuable for practical applica- tions where precise hyperparameter tuning may be challenging.
min)
Accuracy (%)
30
GLEN
Training Time Analysis. Figure 5 presents a comprehensive com- parison of the computational efficiency and accuracy trade-offs across different methods. The scatter plot illustrates the relation- ship between total training time on single NVIDIA A100 80GB GPU for the entire pipeline and achieved accuracy for each approach under 3-shot settings for the ogbn-arxive dataset. The usage of LLMs extends the training time, but our method gains substantial accuracy improvements that justify the computational overhead.
200 Total Time (min)
100 130
2000
250
Figure 5: Training time versus accuracy for all methods.
5 CONCLUSION
In this paper, present GNN-as-Judge, novel framework that addresses the challenge of applying LLMs to sho- sumi-supervised graph learning. Our approach leverages complementary strengths from both GNNs and LLMs through three key mechanisms: (i) subset selection strategy that

---

Published as a conference paper at ICLR 2026
identifies nodes with most information from labeled nodes for pseudo labeling; (ii) a strategic pseudo- label selection that identifies both reliable and challenging nodes, and (iii) a weakly-supervise fine- tuning strategy combining instruction tuning with preference tuning. Experiments across multiple datasets demonstrate that GNN-as-Judge consistently outperforms both GNN architectures and LLLM-based methods, particularly in low-resource settings.
REPRODUCIBILITY STATEMENT
To ensure reproducibility of our results, provide complete source code at https: :tttts::/gii //ggthub. com/ www.01//GNNN-OO. -Judge. Complete implementation details are provided in Appendix C.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 087774, 2023.
Afra Amini, Tim Vieira, and Ryan Cotterell. Direct preference optimization with an offset. arXiv preprint arXiv:2402. 10571, 2024.
Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, al. A general language assistant as laboratory for alignment. arXiv preprint arXiv:21 2021.
Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm understand learning from human preferences. In AISTATS, 2024.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint ArXiv:2204.05800 2022.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICMI., 2009.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In NeurIPS, 2020.
Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. Active learning for graph embedding. arXiv preprint arviv: / 0770055.005.. 2017.
Jonathan Chang and David Blei. Relational topic models for document networks. In AISTATS, 2009.
Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant. In /CML, 2024a.
Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, al. Exploring the potential of large language models (Imss) in learning on graphs. ACM SIGKDD Explorations Newsletter, 2024b.
Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, and Jiliang Tang. Label-free node classification on graphs with large language models (Ilms). In ICLR, 2024c.
Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance- dependent label noise: A sample sieve approach. In NeurIPS, 2020.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In NeurlPS, 2017.
Xinnan Dai, Haohao Qu, Yifen Shen, Bohang Zhang, Qihao Wen, Wenqi Fan, Dongsheng Li, Jiliang Tang, and Caihua Shan. How do large language models understand graph patterns? a benchmark for graph pattern comprehension. In ICLR, 2025.
10

---

Published as a conference paper at ICLR 2026
Jacob Devlin, Ming- Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019.
Kaize Ding, Jianling Wang, Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. Graph prototypical networks for few-shot learning on attributed networks. In CIKM, 2020.
Kaize Ding, Jianling Wang, James Caverlee, and Huan Liu, Meta propagation networks for graph few-shot semi-super learning. In AAAI, 2022.
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.000 2022.
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff. Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. In ICML, 2024.
Yang Gao, Dana Alon, and Donald Metzler. Impact of preference noise on the alignment performance of generative language models. In COLM, 2024.
Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint 180.5v18110005997,, 2018.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024.
Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, and Shi Han. Gpt4graph: Can large language models understand graph structured data? empirical evaluation and benchmarking. arXiv preprint arXiv:2305. 300:::25555.566.. /0066, 2023.
Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Harness- ing explanations: Lim-to-Im interpreter for enhanced text-attributed graph representation learning. In ICLR, 2024.
Jiwoo Hong, Noah Lee, and James Thoree ORPO: Monolithic preference optimization without reference model. In EMNLP, 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. In ICLR, 2022.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.
Zhengyu Hu, Yichuan Li, Zhengyu Chen, Jingang Wang. Han Liu, Kyumin Lee, and Kaize Ding. Let's ask gnn: Empowering large language model for graph in-context learning. In EMNLP Findings, 2024.
Jin Huang, Xingjian Zhang, Qiaozhu Mei, and Jiaqi Ma. Can LLMs effectively leverage graph structural information through prompts, and why? TMLR, 2024.
Kexin Huang and Marinka Zitnik. Graph meta learning via local subgraphs. In NeurIPS, 2020.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b. http::/arriivorrddd///11003333. 2023.
Taehyeon Kim, Joonkee Kim, Gihun Lee, and Se- Young Yun. Instructive decoding: Instruction-tuned large language models are self-refiner from noisy instructions. In ICLR, 2023.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
11

---

Published as a conference paper at ICLR 2026
M Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In NeurIPS, 2010.
Dong-Hy Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop challenges in representation learning. ICML, 2013.
Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg- Scalable agent alignment via reward modeling: a research direction. arXiv preprint arXiv: a 1. 2018.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, 2018.
Yichuan Li, Kaize Ding, and Kyumin Lee. Grenade: Graph-centric language model for self-supervised representation learning on text-attributed graphs. In EMNLP, 2023.
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang.
One for all: Towards training one graph model for all classification tasks. In ICLR, 2024.
Hongrui Liu, Binbin Hu, Xiao Wang, Chuan Shi, Zhiqiang Zhang, and Jun Zhou, Confidence may cheat: Self- Self-training on graph neural networks under distribution shift. In WWW, 2022.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprini 1907. 1692, 2019.
Junyu Luo, Xiao Luo, Kaize Ding, Jingyang Yuan, Zhiping Xiao, and Ming Zhang. Robustfi: Robust supervised fine-tuning for large language models under noisy response. arXiv preprint 2rXiv:2412.14922,, 2024.
Fan Ma, Deyu Meng, Qi Xie, Zina Li, and Xuanyi Dong. Self-paced co-training, In ICML, 2017.
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with reference- free reward. In NeurlPS, 2024.
Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey. arXiv preprint 4rX1:2240..06166 2024.
Subhabrata Mukherjee and Ahmed Hassan Awadallah. Uncertainty- self-training for few-shot text classification. In NeurIPS, 2020.
Galileo Namata, Ben London, Lise Getoor, Bert Huang, and U Edu. Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with graphs, 2012.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. NewrlPS, 2022.
Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford infolab, 1999.
Siyuan Qiao, Wei Shen, Zhishuai Zhang, Bo Wang, and Alan Yuille. Deep co-training for semi- supervised image recognition. In ECCV, 2018.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. In NeurIPS. 2023.
Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. In defense of pseudo- labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. In NeurlPS, 2021.
Prithviraj Sen, Galilco Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad.. Collective classification in network data. Al magazine, 2008
12

---

Published as a conference paper at ICLR 2026
Zeang Sheng, Weiyang Guo, Yingxia Shao, Wentao Zhang, and Bin Cui. Lims are noisy oracles! Ilm-based woise-aware graph active learning for node classification. In KDD, 2025.
Weiwei Shi, Yihong Gong, Chris Ding, Zhiheng MaXiaoyu Tao, and Nanning Zheng. Transductive min-max semi-supervised deep learning using min- max features. In ECCV, 2018.
Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In CVPR, 2016.
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Ander- son. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.174993 2023.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. In NeurIPS, 2020.
Ke Sun. Zhouchen Lin, and Zhanxing Zhu. Multi-stage self-supervised learning for graph convolu- tional networks on graphs with few labeled nodes. In AAAI, 2020.
Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In SIGIR, 2024.
Sheng Wan, Yibing Zhan, Liu Liu, Baosheng Yu, Shirui Pan, and Chen Gong. Contrastive graph poisson networks: Semi-supervised learning with extremely limited labels. In NeurlPS, 2021.
Duo Wang, Yuan Zuo, Fengzhi Li, and Junjie Wu. Lims zero-shot graph learners: Alignment of gnn representations with Ilm token embeddings. In NeurIPS, 2024a.
Haoyu Wang, Shikun Liu, Rongzhe Wei, and Pan Li. Model generalization on text attribute graphs: Principles with large language models. In ICML, 2025.
Hongwei Wang and Jure Leskovec. Unifying graph convolutional neural networks and label propaga- tion. arXiv preprint arXhv:2002.06753 2020.
Jianing Wang, Junda Wu, Yupeng Hou, Yao Liu, Ming Gao, and Julian McAuley. Instructgraph: Boosting large language models via graph-centric instruction tuning and preference alignment. In ACL, 2024b.
Song Wang, Chen Chen, and Jundong Li. Graph few-shot learning with task-specific structures. In NeurlPS, 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022.
Zhihao Wen and Yuan Fang. Augmenting low-resource text classification with graph-grounded pre-training and prompting. In SIGIR, 2023.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplify- ing graph convolutional networks. In ICML, 2019.
Xixi Wu, Yifei Shen, Fangzhou Ge, Caihua Shan, Yizhu Jiao, Xiangguo Sun, and Hong Cheng. When do Ilms help with node classification? a comprehensive analysis. In ICML, 2025.
Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In ICML, 2018.
Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. Network representation learning with rich text information. In IJCAI, volume 2015, 2015.
Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit Singh, Guangzhong Sun, and Xing Xie. Graphformers: Cenn-eested transformers for representation learning textual graph. In NeurlPS, 2021.
13

---

Published as a conference paper at ICLR 2026
Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. ICML, 2016.
Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang. Language is all graph needs. In EACL, 2024.
Jianxiang Yu, Yuxiang Ren, Chenghua Gong, Jiaqi Tan, Xiang Li, and Xuecang Zhang. Empower text-attributed graphs learning with large language models (Ilms). In AAAI, 2025.
Wentao Zhang, Yexin Wang, Zhenbang You, Meng Cao, Ping Huang, Jiulong Shan, Zhi Yang, and Bin Cui. Rim: Reliable influence-based active learning on graphs. In NeurIPS, 2021.
Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. Learning large-scale text-attributed graphs via variational inference. In ICLR, 2023.
Siyan Zhao, John Dang, and Aditya Grover. Group preference optimization: Few-shot alignment of large language models. In ICLR, 2024.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily graph neural networks: Current limitations and effective designs. In NeurlPS, 2020.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
This Appendix provides comprehensive supplementary material for the submitted paper including detailed information across the following sections:
Section A: Dataset statistics and descriptions including comprehensive analysis of evaluation
datasets with node/edge counts, feature dimensions, and label space characterization.
Section B: Comprehensive baseline method descriptions covering Classical GNN Models, LLM- as-Predictors approaches, and state-of-the-art LLM-Graph integration methods.
Section C: Implementation details for our approach and all baselines, including hyperparameter settings, training configurations, and computational environment specifications for reproducibility.
Section D: Theoretical analysis with complete proofs for Theorem and Theorem
Section E: Detailed prompt templates used for L.LM-based methods including our GNN-as- Judge framework and baseline approaches.
Section F: Quantitative analysis of pseudo-label selection including agreement patterns, accuracy
distributions, diversity metrics, and selection strategy comparisons.
Section G: Additional experimental results covering sensitivity analysis, integration with various LLMs, performance heterophilic graphs.
Section H: Discussion of limitations and potential future research directions for the GNN-as-Judge framework.
Section I: Extended related work.
Section J: Disclosure of Large Language Model usage in manuscript preparation for transparency and reproducibility standards.
A DATASET
A.1 DATASET STATISTICS
Table 3: Summary statistics of the evaluation datasets.
Dataset
Nodes
Edges
Features
Classes
Cora
2,708
5,429
1,433
Citeseer
3,327
4,732
3,703
6
Pubmed
19,717
44,338
500
ogbn-arxiv
169,343
1,166,243
128
40
oglon-product
54,025
74,420
100
47
Table 3 presents the detailed statistics of the datasets used in our experiments, including the number of nodes, edges, features, and classes for each dataset.
A.2 DATASET DESCRIPTIONS
Cora (Yang et al., 2016). The Cora dataset comprises 2,708 scientific publications classified into one of seven machine learning research categories. The citation network consists of 5,429 links, where papers were selected such that every paper citeps or citepd by at least one other paper in the final corpus.
citeseer (Sen et al., 2008). The citeseer dataset contains 3,327 computer science papers classified into 6 categories across different CS research areas. The citation network includes 4,732 links, with each paper represented by -i0eeeiimss features derived from bag-of-worde representation of the paper content.
Pubmed (Sen et al., 2008). The Pubmed dataset consists of 19.717 scientific publications from the PubMed database pertaining to diabetes research, classified into one of three diahetes-related
15

---

Published as a conference paper at ICLR 2026
categories. The citation network consists of 44,338 links, with each paper represented by 500- dimensional features extracted from the paper abstracts.
ogbn-arxiv (Hu et al., 2020). The ogbn-arxiv dataset is directed graph representing the citation network between computer science arXiv papers indexed by Microsoft Academic Graph (MAG). Each node represents an arXiv paper, and each directed edge indicates that paper citeps another.
ogbn-products (Hu et al., 2020). The ogbn-products dataset represents an Amazon product co-purchasing network with product descriptions as raw text. Nodes represent products sold on Amazon, and edges between products indicate co-purchasing relationships. We use subset of 54,025 products for computational efficiency following previous work (He al., 2024) while maintaining the dataset's key characteristics.
A.3 LABEL SPACE
Within each dataset, nodes are labeled according to their academic category, as detailed in Table 4. For example, the arXiv dataset includes 40 computer science sub-categories such as cs.Al (Artificial Intelligence) and cs (Databases).
Table 4: Label spaces of the datasets used in our experiments.
Dataset
Label Space
Cora
Rule Learning. Neural Networks, Case Based, Genetic Algorithms, Theory, Reinforcement Learning, Probabilistic Methods
citeseer
Agents, ML (Machine Learning), IR (Information Retrieval), DB (Databases),
HCI (Human-Computer Interaction). Al (Artificial Intelligence)
Pubmed
Experimentally induced diabetes, Type diabetes, Type diabetes
ogbn-arxiv
Ns. cs.MM, es. cs. CY, cs. CR, cs. DC, es. cs. ss,C,, cs.NI, cs.CC, cs.Al. cs.MA, es. cs.NE, es. es.AR, cs. CV. Cs. ET. es.SY, cs.CG, OH. cs.PL, cs.SE, cs.LG, cs.SD, cs.SI, Rs, cs.IT, cs.PF, cs.CL, cs.,R, MS. cs. DS. Cs.OS, CS. DB. DI. DM.
productss
Home & Kitchen, Health Personal Care, Beauty, Sports & Outdoors, Books, Patio Lawn & Garden, Toys & Games, CDs & Vinyl, Cell Phones & Accessories, Grocery & Gourmet Food, Arts Crafts & Sewing, Clothing Shoes & Jewelry, Electronics, Movies & TV, Software, Video Games, Automotive, Pet Supplies, Office Products, Industrial & Scientific, Musical Instruments, Tools & Home Improvement, Magazine Subscriptions, Baby Products, Appliances, Kitchen & Dining, Collectibles & Fine Art, All Beauty, Luxury Beauty, Amazon Fashion, Computers, All Electronics, Purchase Circles, MP3 Players & Accessories, Gift Cards, Office & School Supplies, Home Improvement, Camera & Photo, GPS & Navigation, Digital Music, Car Electronics, Baby, Kindle Store, Kindle Apps. Furniture & Decor
B BASELINE METHODS
To evaluate the effectiveness of our proposed framework, we compare it against established methods from three primary categories: (1) Classical GNN Models, (2) Graph Methods. These baselines represent diverse approaches to handling Text-Attribtted Graphs, ranging from traditional graph neural networks to modern large language model-based methods.
and (3) LLM-
B. CLASSICAL GNN MODELS
We include established graph neural network architectures that rely primarily on graph structure and node features without leveraging the reasoning capabilities of language models:
16

---

Published as a conference paper at ICLR 2026
GCN (Kipf& & Welling, 2017): Graph Convolutional Networks perform neighborhood aggregation through spectral convolutions, using a simple and effective message- passsengaainggggg scheme.
SGC (Wu et al., 2019): Simple Graph Convolution simplifies GCNs by removing nonlinearities between layers and collapsing weight matrices, resulting in a single linear transformation followed by softmax classifier, while maintaining competitive performance with significantly reduced computational complexity.
B.2 LLM-AS-PREDICTEO METHODS
We include baselines that utilize general-purpor LLMs with various prompting strategies for direct downstream classification tasks. In this paradigm, node's 's textual and structural information, along with task-specific instructions, are tokenized and input into an L.M for prediction:
Direct application of LLMs without task-specific training, relying on the model's pre-trained knowledge.
Zero-shot Prompting:
Graph Chain-offThhogeee (Wei et al., 2022; Chen et al., 2024b): Enables step-by-step reasoning by breaking down complex graph-related problems into sequential reasoning steps.
Neighbor-Augmenle Prompting (Chen et al., 2024b): Enriches prompts with structural infor- mation from node neighborhoods, providing context about graph topology enhance zero-shot performance.
B.3 LLM-GRAPH METHODS
Aligning with our focus, we benchmark against various state-of-the-ar methods that effectively integrate LLMs with graph neural networks for enhanced performance Text-Attributed Graphs:
GLEM (Zhao et al., 2023): Combines language models with graph neural networks through unified framework that leverages both textual and structural information.
TAPE (He et al., 2024): A text-attributed graph learning framework that effectively harnesses the power of language models for graph understanding.
LLM-GNN (Chen et al., 2024c): Integrates large language models with graph neural networks for improved node classification performance.
LLaGA (Chen al., 2024a): Employs multi-step approach where node text is first encoded via language model, then processed through a GNN, concatenated across layers, projected into the LLM's dimensionality, and finally combined with instructions for label prediction. Only the projection layer parameters are tuned using next-token prediction
next-token-prediction loss.
GraphGPT (Tang et al., 2024): Implements comprehensive framework with three distinct pre-training and instruction tuning stages to effectively integrate graph structure with language understanding.
C IMPLEMENTATION DETAILS
This section provides comprehensive details about the implementation, hyperparameter settings, and training procedures used in our experiments.
C.1 IMPLEMENTATION DETAILS FOR GNN-AS-JUDGE
Data Splits. For the Cora, ci citeseer, and Pubmed datasets, we follow standard node classifi- cation protocols using 3-shot, 5-shot, and 10-shot settings, where n-shot refers to n labeled nodes class for training. We use 500 nodes for validation and randomly select 1,000 nodes for testing from the remaining nodes. For ogbn-arxiv and oognnpprodue
dataset, we adopt the original split for both validation and testing provided by (Hu al., 2020). For the training split, randomly select 3-shot, 5-shot, and 10-shot settings by sampling a nodes from each class within the original training set to create labeled node sets.
GNN Component. For the GNN component our GNN-as-Jude -uWdas-Judge framework and other GNN-based baselines, we implement 2-layer GCN architecture (Kipf & Welling, 2017) with 64-dimensional
17

---

Published as a conference paper at ICLR 2026
hidden representations. We perform limited grid search dropout rates, exploring values in [0.3, 0.5, 0.7], and include batch normalization in our architecture. We set the learning rate to le-2 and train for up 200 epochs with patience of 100 for early stopping. For optimization, Adam optimizer with a weight decay of 5e-4, following standard practices in graph neural network training.
use the
LLM Component. Our GNN-as-Judge framework utilizes -lama-3-8B-- ---t-tttttttttt (Grattafiori et al., 2024) as the base model. We implement parameter-efficient fine-tuning using LoRA (Hu al., 2022) with rank 8 and alpha value of 16, which enables efficient adaptation of the pre-trained weights while maintaining computational feasibility. We set the dropout rate to 0.1 and use batch size of 8.
Instruction Tuning Configuration. We learning rate of 10-6 for the Llama-3 model and train for 10 epochs during the instruction tuning stage.
weakly-Supervised Fine- Tuning Configuration. We maintain a learning rate of x 10-6 with 8 epochs for all datasets. The hyperparameter À in Eq. which balances instruction tuning and preference tuning losses, is set to 0.1 across all datasets and settings based on our parameter sensitivity analysis.
Selection Hyperparameters. For our proposed GNN-as-Judge pseudo-label selection mechanism described in Section 3, we employ several key hyperparameters. We set the top-K selection parameter to 1,500 influential nodes for all datasets to balance computational efficiency with pseudo-labeling coverage. For large-scale datasets (ogbn-arx arroin i,, labeled nodes. The preference score threshold is set 0.7 across all datasets.
we extract subgraphs around
Computational Environment. All experiments are conducted on NVIDIA A100 GPUs with 80GB memory.
C.2 IMPLEMENTATION DETAILS FOR OTHER BASELINES
GLEM (Zhao et al., 2023): Following previous work (Wu et al., 2025), we set the number of EM iterations I and the eseudo-labeling ratio to 0.5. For the GNN module within GLEM, we use layers with 64-dimensional hidden representations. For the LM module, we use RoBERTa (Liu al., 2019) as the base model with LoRA optimization and a batch size of 32. The LM is pre-trained first across all datasets before joint training with the GNN component.
TAPE (He et al., 2024): We utilize the provided prompt templates from the original paper to generate explanations using -lama-3-8B-Instruu (Grattafiori et al., 2024). We use RoBERTa (Liu et al., 2019) as the language model component, which is fine-tuned using LoRA with default parameter settings, while the GNN configuration remains consistent with our method.
LLM-GNN (Chen al., 2024c): To ensure fair comparison with this baseline, we adapt the original zero-shot approach to our few-shot setting. We use instruction- Instruct (Grattafiori al., 2024) on labeled data as the annotator. We employ the DA-AGE method proposed in the original paper for pseudo-label selection and train the GNN using both labeled data and pseudo-labeled data together.
Llama-3-8B-
LLaGA (Chen et al., 2024a): We use HO templates for all experiments with the number of hops set to 4. We use RoBERTa (Liu et al., 2019) as the text encoder. The linear projection layer do (-) consists of a 2-layer MLP with hidden dimension of 2048. The batch size set to 64 and learning rate to 1 X 10-4 The number of training epochs is set to 10 for all settings.
GraphGPT (Tang et al., 2024): Following previous work (Wu et al., 2025), we exclude the text- graph grounding stage since the inclusion this stage does not consistently lead to performance improvements. For the self-supervised training data for each dataset to perform dataset-specific graph matching tasks. In the task-specific instruction tuning stage, we utilize the training data to create (instruction, ground-truth label) pairs following the original prompt design. The training parameters for supervised instruction tuning stage include 2 epochs with a learning rate of 1 x 10 and batch size of 16. For task-specific instruction tuning stage, we train for 10 epochs with learning rate of 1 x 10 and a batch size of 32.
instruction tuning stage, we construct self-supervised
18

---

Published as a conference paper at ICLR 2026
D THEORETICAL ANALYSIS
D.1 PROOF OF THEOREM 1
Following previous study (Xu et al., 2018; Huang Zitnik, 2020; Wang & Leskovee, 2020), we GCN (Kipf & Welling, 2017) the exemplar GNN model. The propagation mechanism for the 1-th layer in GCN is formulated H(++) a oÁÂT ((O),, where H represents the node feature matrix and (!)) denotes the learnable parameter matrix layer l. The term À D-1A corresponds to the row-normalized adjacency matrix. To simplify our theoretical derivations, we adopt the standard assumptions from (Wang & Leskovec, 2020), that the activation function o as the identity operator and the weight matrix W is the identity matrix.
Theorem 1. Let Per,e, denote the set of all paths between nodes e; and V. and let Pic.os S Porce denote the set of shortest paths. For any path Pen,,y let DiM sa be the geometric mean of node degrees occurring on path t defined as D&M (IIngr DULus) 1/10| where [t] is the path length and Dunn. .. is the degree of node vt. Define D&M = minkeP:,,,, (DaM) } as the minimum geometric mean among shortest paths, ht d(e;, Uz} the shortest path distance between us and v, and (P as the number of shortest paths between vs and Vy. Then, the node influence Iv,,y from v to
2j satisfies:
Luses = -10xx 10x( S
(9)
(DoM))
Proof. According to the GCN propagation rule, the final representation of node vj is:
0ujox (20)
(1)
Dojus vLEN(())
where (W²)) denotes the neighboring nodes of ej, Geyor is the edge weight (typically for un- weighted graphs), and Daye, is the degree node Uy.
By recursive substitution incorporating neighbors at multiple hops:
(20) Dojus vLEN(1s)
Σ
1
Σ
(20)
Onyvn DeLie ((E)
Ougen
Ovn Uo Po
Dem voEN(Um))
[x((x)) dxc! (cc)
The node influence Ier,,
can be computed by taking the derivative with respect to
(co) Since we are calculating the gradient between two feature vectors Xn; Xa:;
(oo) and Xa; (co) the partial
derivative on nodes that are not in the paths PI..... Pm between node 2ù and by becomes and these nodes are removed:
xx(20)
Σ
Ox! (20)
Dejos
Ousipt
Dpt
Optin4
Dpt ph.
Oxl,
(x0)
(3)
IEPuyinG
We separate the scalar terms and the derivative term within the matrix norm and use the absolute homogencous property (||aA|| = |a]]|A|] of the matrix norm:
(00)
Ougant Opt
(x((0)
Degon Dpt
Dpt. ph,
xx0)
BEPrging
(xe(
Since the Jacobian of the same vectors
is the identity matrix and for any subordinate norm axc
(I) ||| sip A2]|))))) we have ||7||
19

---

Published as a conference paper at ICLR 2026
I 1ngay
Σ
Ouy.pt Pii-p
(5)
Dojoz Pri
Pah, ph:
MEPuyi
Using the triangle inequality and identifying the maximum term among all paths:
Iu,,, S
Σ
aug.p Opi.ph api,
(6)
Dosoz pi
Dyn, the
LEP tePogive
$ Poy.v.l
max PePrgi
Ou pt Ppi
aph,
Desoz Drin
Drh, pha
For binary networks where edge weights a (non-negative), and focusing on the path p" that maximizes the influence contribution:
Tonny S
(Poyiul Dejuy Doj" .f"
(8)
Expressing the degree terms in geometric mean format where |6*| is the length of path t*:
Tuies S IP iPor.osl
(9)
(arr))
D.2 PROOF OF THEOREM 2
Theorem 2. Consider Vsetected with ground truth labels y". Let PLLM and PONN denote the individual accuracies of the LLM and GNN respectively. Under conditional independence of model errors due different inductive biases and uniform error distribution across incorrect classes, the accuracy of the agreement set satisfies:
PLLM PGNN
Plu U4 Vagreed)
(10)
(1-Prr) Prwv
PLIM PGNN +
C-1
where the number of classes. Furthermore, this lower bound exceeds max(PLLM PONN) when both models perform better than random guessing.
Proof. We establish lower bound on the accuracy of the agreement set by analyzing the probability of correct predictions under model agreement.
The probability that both models agree on their predictions decomposes as:
P(agreement) PM y/ .NN F() P
yt GNN + vi) PLLM PGNN (11)
where represents the probability of agreeing
incorrect prediction.
Under conditional independence and uniform error distribution across incorrect classes:
Σ
P(y LLM j/y/ L.M + Fv)) P(y GNN = -lys GNN + vt) x Lm + ut) P(y GNN + vt)
(12)
PLIM) PONN)
Σ ΣE_ C-1 (1 PLIM) (1 PANN) jf
(13)
0-1
By Bayes' theorem, the accuracy of the agreement set is:
P(y lagreement) = P(agreement y/) P(yf)
PLLM PONN
(14)
9 (agreement)
PLIM PONN +
20

---

Published as a conference paper at ICLR 2026
PLLM PONN
(15)
PLLM PUNN (-PLLM)(L PONN]
This establishes the lower bound stated in the theorem.
To show this lower bound exceeds individual model performance, we prove it for PLLM
PLLM PONN
PLLM PONN + (1-pux) (1-PNN) >
PLIM
(16)
C-1
Cross-multiplying and simplifying:
PLLM) PONN)
PLLM PONN > PLLM PPLMM PONN +
(17)
C C-1
PLLM PGNN PLIM PONN + PLLM PLLM)(1 -PPGN)
(18)
C-1
PLLM PGNN (1 PLLM) PLIM(1 PLIM)(1 PONN)
(19)
C-1
Dividing by (1 - PLLM) > 0:
1 PONN
1 C
PUNN
C-1
PC 1) 1 PONN PUNN C>1 PONN
(20)
By symmetry, the same holds for PLLM C. Therefore, when both models perform better than random guessing, the lower bound exceeds both individual accuracies:
P(y lus Vagreed)
PLIM PGNN
-PP (1- max(PLIM,
(21)
PLLM PONN +
E PROMPTS
This section provides detailed prompt templates used for all LLM--ased methods in our experiments. We use (raw text) to denote the node's original textual content, (labels) to represent the datasct- specific label space, and (graph) for tokenized graph context when applicable.
E PROMPTS FOR GNN-AS-JURDE
For our proposed method, we use straightforward insllwtiion-oolloiing prompts tailored to each dataset:
21

---

Published as a conference paper at ICLR 2026
GNN-as-Judge Prompt Templates
Given node-centered graph with centric node description: (raw_tess fraw_text), each node represents a paper, we need to classify the center node into classes: Case Based, Genetic Algorithms, Neu- ral Networks, Probabilistic Methods, Reinforcement Learning, Rule Learning, Theory, please tell me which class the center node belongs to?
Cora:
citeseer: Given a node-centered graph with centric node description: (raw_text), each node represents paper, we need to classify the center node into 6 classes; Agents, ML (Machine Learning), IR (Infor- mation Retrieval), DB (Databases), HCI (Human- Computer Interaction), AI (Artificial Intelligence), please tell which class the center node belongs to?
Pubmed: Given node-centered graph with centric node description: (raw_text), cach node represents paper about Diahetes, we need to classify the center node into 3 classes: Experimentally induced diabetes, Type diabetes, Type 2 diabetes, please tell which class the center node belongs to?
ogbn-arxiv: Given node-centered graph with centric node description: soww_tex),, we need to classify the center node into 40 arXiv CS sub-calegories: cs.Al( (rrtffiill Intelligence), cs. CV(Computer Vision and Pattern Recognition), Gaachhne Learning), CL(Computation and Language), NENuurl Evolutionary Computing), .... please tell me which class the center node belongs to?
ogbn-products: Given node-centered graph with centric node description: (raw_text), each node represents product, need to classify the center node into 47 classes: Home & Kitchen, Health & Personal Care, Beauty, Sports & Outdoors, Books, Electronics, ... please tell which class the center node belongs to?
E.2 PROMPTS FOR GRAPHGPT
GraphGPT uses graph-aware templates with structured graph tokens:
GraphGPT Prompt Templates
Citation Networks (Cora, citeseer, Pubmed, ogbn-arxiv):
Given citation graph: (graph} where the 0-th node is the target paper, with the following information: (raw text). Question: Which
following specific research does this paper belong to: (labels).
Directly give the full name of the most likely category of this paper.
E-commerce Network (ogbu-products):
Given e-commerce network: (graph) where the node is target product, with following information: (raw_text). Question: We need to classify the center product into 47 classes: (labels). Directly tell me which product category the center product belongs to.
E.3
PROMPTS FOR LLAGA
LLaGA employs HO templates with multi-hop structural information:
LLaGA Prompt Templates
Citation Networks (Cora, citescer, Pubmed, ogbn-arxiv):
Given a node-centered graph: (graph), each node represents paper, we need to classify center node into [N] classes: (labels), please tell me which class the center node belongs to?
E-commerce Network (ogbu-products):
Given a node-centered graph: (graph), each node represents a product, we need classify the center node into 47 classes: Home & Kitchen, Health & Personal Care, Beauty, Sports & Outdoors, Books, Patio Lawn & Garden, Electronics, plcase toll me which class the center node belongs to?
E.4 PROMPTS FOR ZERO-SHIOT LLM METHOOS
For direct LLM inference without fine-tuning, employ several prompting strategies:
22

---

Published as a conference paper at ICLR 2026
Zero-Shot Prompting Templates
Chain-of- Thought:
Thain-of-Thoughtt:
Question: Which of the following types does this |paper/product) belong to? Here are the [N] categories: (labels). Let's think about it step by step. Analyze the content of the node and choose one appropriate category. Output format: (reason:). (classification:)
Crighbor-Augmenter
Given the information of the node: (raw_texi). Given the information of its neighbors (raw_ text). Here I give you the content of the node itself and the information of its 2-hop neighbors. The relation between Question: Based this information. which of
the node and its neighbors is I"citation"/" "arooa""""oopphaseee the following categories does this [item] belong to? Here are the [N] categories: (labels). Reply only one category that you think this fitem] might belong to.
F QUANTITATIVE ANALYSIS OF SELECTED PSEUDO LABELS
To provide deeper insights into our GNN-as-Judg! framework, we conduct a comprehensive analysis of the pseudo-label selection process across all datasets. This analysis examines the agreement patterns between LLM and GNN predictions, the accuracy distributions of selected labels, and the composition of final augmented training sets. Table 5 presents the quantitative results of our pseudo-label selection strategy in the 3-shot setting.
F.1 AGREEMENT AND DISAGREEMENT PATTERNS
Table 5: Quantitative analysis of pseudo label selection for the 3-shot setting.
Initial Pool
Final Set
Dataset
A/D Ratio
Agree Acc.
Disagree Acc.
Size
Final Acc.
Sel. Disagree Acc.
Cora
978/520
93.35
61.68
1,103
92.11
82.41
citeseer
881/614
91.59
66.56
1,120
88.83
72.17
Pubmed
991/504
92.02
31.55
1.191
83.29
40.03
ogbn-arxiv 1/888
84.40
37.38
817
78.09
59.37
ogbn-products 753/733
87.12
37.24
883
81.99
52.29
Table 5 presents the quantitative results of our pseudo-label selection strategy in the 3-shot setting (seed=42). The table reveals several key patterns in how LLMs and GNNs agree or disagree on different datasets. The agreemen//lisgrremeen (A/D) ratios vary across datasets. Agreement nodes consistently achieve high accuracy, validating our theoretical analysis in Theorem 2. Notably, the disagreement accuracy is much lower and varies substantially across datasets, highlighting the importance of careful selection within the disagreement set.
Table 6: Accuracy (%) of our final selected pseudo-label set compared baseline selection methods of the same size.
Dataset
GNN Pred. Acc.
LLM Pred. Acc.
Sel. Acc.
Cora
82.32
68.44
92.11
citesser
81.07
60.80
88.83
Pubmed
71.20
82.61
83,29
ogbn-arxiv
55.93
50.55
78.09
ogba-products
61.38
62.85
81.99
To further contextualize our framework's performance, Table 6 compares the accuracy of our final selected pseudo-label set "Sel. Acc") against baselines that randomly select set of the same size using only GNN or only LLM predictions. The results clearly indicate that our method consistently outperforms both individual models across nearly all datasets.
23

---

Published as a conference paper at ICLR 2026
Dataset
Pearson
Arjo
Aoji.
Cora
0.258
0.216
0.162
Citeseer
0.202
0.159
0.115
Pubmed
-0.048
0.020
0.062
ogbn-arxiv
0.117
0.047
0.058
ogbn-products
0.140
0.094
Table 7:
P(LLM GNN em) P(LLM em) and AGjL P(GNN LLM err) P(GNN err). Smaller values indicate weaker dependence.
Error independence diagnostics. ALG
Table 8:
Effect of preference score threshold т on disagreement set size and GNN accuracy. Percent- ages denote proportion of the full disagreement set.
Dataset
Set Size
% of Full
Accuracy
0.10
471
90.6%
63.27%
0.40
262
50.4%
75.95%
Coza
0.70
123
23.7%
82.11%
0.90
35
6.7%
82,869
0.10
586
95.4%
67.64%
0.40
427
69.5%
71,77%
Clleseer
0.70
238
38.8%
72.27%
0.90
89
14.5%
75,28%
0.10
463
91.9%
31.17%
0.40
325
64.5%
36.63%
Pubmed
0.70
200
39.7%
40.03%
0.90
51
10.1%
$2.94%
0.10
815
91.8%
39.37%
0.40
458
51.6%
50.76%
ogbn-arxiv
0.70
205
23.1%
59.51%
0.90
76
8.6%
63.16%
0.10
625
85.3%
40.22%
0.40
298
40.7%
47.67%
ogbn-products
0.70
130
17.7%
52.31%
0.90
58
7.9%
63.79%
F.2 ERROR CORRELATION ANALYSIS
We quantify error dependence using three complementary metrics:
Pearson correlation between binary enror indicators (1 error), capturing linear dependence.
ALGG = P(L=1 G=1) - P(L=1): the change LLM error probability when the GNN errs.
Positive values indicate the LLM is more likely to when the GNN errs.
AGp. = P(G=1 L=1) P(G=1): the change GNN error probability when the LLM errs.
Across all datasets, the dependence metrics remain small, indicating that GNN and LLM errors exhibit only weak correlation.
F.3 DISAGREEMENT SET SIZE VS. QUALITY
To analyze the trade-off between disagreement set size and quality, we examine how different preference score thresholds (r) affect the filtered disagreement set. Table shows the relationship between set size and GNN accuracy across datasets. Across all datasets, we observe consistent pattern: higher filtering (smaller sets) yields substantially higher accuracy.
F.4 SELECTION STRATEGY COMPARISON
The results in Table 9 reveal several important insights about pseudo label selection strategies. To ensure fair comparison, all methods follow the same experimental protocol: first selecting 1,500 candidate nodes using the respective selection strategy, then annotating them using Llama3-8B.
24

---

Published as a conference paper at ICLR 2026
Table 9: Comparison of different pseudo label selection strategies selected pseudo labels accuracy
(%). Best results are bolded.
Method
Cora
citeseer
ogbn-arxiv
Random
64,4
60.80=2.115
53.93-+1.87
Degree
1.08
64.26:1.89
50.2412.86
AGE
67.36=1.74
65.78
52.31=1145
Confidence
75.71
1.779
66.141148
LLM-GNN
76.49::0.4
12.34
54.6711.53
Instruct trained under 3-shot setting. The results demonstrate that traditional selection methods face significant limitations when LLMs are unreliable. Graph topology-based approaches achieve relatively low accuracy, indicating that structural information alone is insufficient for identifying nodes suitable for high-quality pseudo-labeling using LLMs. Interestingly, even LLM confidence-based selection cannot fully ensure high-quality pseudo labels. This highlights a fundamental challenge: when the underlying LLM is unreliable in low-resource settings, neither graph structure model confidence provides adequate signals for pseudo label selection. These findings motivate our GNN- as-Judge approach, which addresses these limitations through collaborative filtering that leverages both structural and semantic information.
F.5 DIVERSITY ANALYSIS OF SELECTED NODES
To understand the diversity of nodes selected by our approach, we conduct an analysis to demonstrate the diversity and representativeness of nodes selected by our method. From Table we observe that LLM confidence is a strong baseline for selecting high-quality pseudo labels. While pseudo label accuracy is crucial, selecting diverse nodes equally important for effective L.LM training, as prevents overfitting to specific node types and improves generalization. We compare our approach with using I.I.M confidence to select the top 1 1,500 unlabeled nodes for the ogbn-arx axxiv dataset, extracting embeddings using Llama3-88 -inma-----nnstrect.... Instruct.
(a) LLM Confidence
(b) GNN-as Judge
Figure 6: t-SNE visualization of selected pseudo-labeled nodes the ogbn-arxiv dataset. Colors represent different classes.
Figure provides compelling visual comparison of the selection strategies. The LLM Confidence approach shows distinct, tight clusters with strong class separation, indicating it heavily favors nodes from a limited subset of the feature space. In contrast, our CNN--as-udge approach selects nodes that are more broadly distributed across the entire feature space, covering wider variety of node types and characteristics. This broader selection ensures that the I.I.M receives training examples from diverse areas of the data distribution, leading to better generalization across different node patterns.
Table 10 quantifies the diversity advantages our approach. Coverage measures the average distance from each class centroid to the nearest selected node. Representativenes: quantifies how well the selected nodes represent the overall data distribution using the Wasserstein distance between selected and full node distributions. Intra-class Variance measures the diversity within each class among selected nodes. This demonstrates that our approach selects nodes that are more representative the overall data distribution.
25

---

Published as a conference paper at ICLR 2026
Table 10: Quantitative diversity comparison between selection methods on oobnn arxiv dataset.
Method
Coverage
Representativeness
Intra-class f
Confidence
11.62 8.97
0.34 0.10
12.52 15.29
GNN-as-Judes Judge
G ADDITIONAL EXPERIMENTS
G.1 SENSITIVITY TO
In our approach, the hyperparameter À controls the balance between standard instruction tuning and preference tuning as shown in Eq. 5. Figure 7 (left) illustrates how different À values affect performance on Cora, citesees and Pubmed datasets in 3-shot settings. We observe clear trend where lower values yield significantly better results, with performance declining as increases. This suggests that using moderate weight for preference tuning loss will maintain better model performance.
(h)
(h)
Caru
El
Ciisnee
Pubtard
Accuracy (50
80-
80
75
70-
70
Preforeuce Tuslag Mothod
Figure 7: (a) Sensitivity analysis of the hyperparameter À showing performance trends across different values. Lower À values generally yield better performance, particularly for Cora and citeseer. (b) Performance comparison across different preference tuning losses (DPO, SimPO, ORPO), demonstrating the framework's compatibility with various optimization methods.
G.2 ADAPTATION TO DIFFERENT PREFERENCE TUNING LOSSES
We evaluate the performance of our GNN-as-Judge framework across different preference tuning losses, comparing DPO (Rafailov et al., 2023), SimPO (Meng et al., 2024), and ORPO (Hong al.. 2024) on Cora, citeseer and Pubmed in 3-shot setting. As shown in Figure (right), these preference tuning methods demonstrate relatively comparable performance. This observation highlights that our GNN-as-Judge a alug-and-plas framework where different preference tuning losses can be seamlessly integrated without compromising overall performance.
G.3 EFFECTIVENESS OF INFLUENTIAL NODE SELECTION
To validate the importance of our influential node selection strategy, we compare the quality of pseudo labels generated for different node selection approaches. Specifically, examine: (1) random selection of 1,000 unlabeled nodes, and (2) our influential node selection according to node incluence score. Table 11 presents the pseudo-label accuracy when using either GNN or LLM predictions as pseudo labels.
G.4 INTEGRATION WITH VARIOUS LLMS
To demonstrate the generalizability our GNN-as-Judge framework, we evaluate its performance with different backbone LLMs. Table 12 presents results using Distral-BB-Instruee (Jiang et al., 2023) and Llama3- B-aa33----Imae 8B-Instruct (Grattafiori et al., 2024) under the 3-shot setting, comparing their zero-shot baseline performance against our framework.
The results in Table 12 demonstrate the remarkable robustness of our GNN-as-Judge framework across diverse LLM families. Despite significant variations in the baseline zero-shot capabilities
26

---

Published as a conference paper at ICLR 2026
Table 11: Comparison of pseudo-label accuracy (%) for different node selection strategies on 1,000 unlabeled nodes under 3-shot setting. Higher accuracy indicates better quality of selected nodes for useudo-labeling.
Random Selection
Infiuential Selection (Ours)
Dataset
Ace.
LLM Ace.
GNN Ace.
LLM Acc.
Cora
68.244 1.83
66.92:22.14
82.39 26
68.84:1..5
citeseer
81.92.1
Pubmee
65.43=1.33
16.26:1:119
71.56:10.94
83.42:10.87
ogba azxiv
38.92-22.45
52.37:22.11
56,51=1.995
51,16:22.12
ogbn products
60.15=1194
73.83:::176
1..54
62.91
Table 12: Performance comparison of different LLMs integrated with our GNN-as-Judge framework under 3-shot setting. Results show accuracy (%) with standard deviation.
LLM
Method
Cora
citeseer
Pubmed
ogbn-arxiv
agbn-products
Zero-Shot GNN-as-Judge
19.78=1.226
42.36=50.33
77.24=1.15
43.06=::244
50.80
Mistral-7B
76.16:1:1.15
73.08-10.89
84.79=0.94
57.36:::152
79.02:1141
Zero Shut
65.54.10.26
0..67.1..64
74.51 00039
50.18.1 1.44
75.48:1.54
Llama3 SB
GNNas-Judge
87.12:
62.21
of these models, our framework consistently elevates performance competitive levels across all architectures.
G.5 PERFORMANCE ON HETEROPHILIC GRAPHS
To evaluate the effectiveness of our GNN-as-Judge framework on heterophilic graphs where connected nodes tend to have different labels, we conduct experiments on Cornell and Wisconsin (Wang et al., 2025) datasets using H2 GCN (Zhu et al., 2020) the GNN backbone. These datasets exhibit low homophily ratios (0.1 for Cornell 1 and 0.16 for Wi sisonnssn) presenting unique challenges where traditional GNN assumptions of homophily do not hold. While our primary focus is not
Table 13: Performance comparison heterophilic graphs (Cornell and Wi Wisconsin) using
H₂GCN as GNN backbone under 3-shot setting. Results show accuracy (%) with standard deviation.
Method
Cornell
Wisconsin
Hy HCN
41.15113.12
Zero-Shox
79.84 .22
13.49::1.35
GNN-as.Judge
84.65 1.23
76.95 2.00
heterophilic graphs, these results demonstrate that our framework can seamlessly adapt to different GNN types without requiring architectural modifications. The consistent performance improvements over both the GNN baseline and zero-shot LLM approach validate the broad applicability of our collaborative pseudo-labeling strategy across diverse graph characteristics and GNN architectures.
G.6 MEMORY USAGE ANALYSIS
Figure 8 presents comprehensive comparison of the memory efficiency and accuracy trade-offs across different methods. The scatter plot illustrates the relationship between peak GPU mem- ory consumption during training and achieved accuracy for each approach under 3-shot settings on the ogbn-arxiv dataset us- ing single NVIDIA A100 80GB GPU. LLM-based approaches require substantially higher memory usage. The analysis reveals clear accuracy memory trade-off, where methods achieving higher performance generally require more computational resources due to the integration of large language models.
GNN judgo GBI
80-
Accuriccy (%)
TAPY
G.3 col
20
memory
Memory (GB)
Figure 8: Memory usage versus accuracy.
27

---

Published as a conference paper at ICLR 2026
G.7 INTEGRATION WITH DIFFERENT GNN BACKBONES
To assess the architectural robustness of the proposed framework, we evaluate its performance when paired with several widely used GNN backbones, including GCN, GAT, and GraphSAGE. Table 14 reports the performance across three representative datasets under 3-shot setting. The results show that the framework achieves consistent improvements regardless of the backbone used, suggesting that the method integrates well with a variety of GNN architectures.
Table 14: Performance of the proposed framework when integrated with different GNN backbones. Results are reported as mean accuracy standard deviation over three runs.
Architecture
Citeseer
Cora
ogbn-arxiv
GCN
73.59
77.89
7.424
GAT
0.71
79.12 = 0.62
70.45 +: 0,51
GraphSAGE
74.71 1:
77.67 1:
1.04
H LIMITATIONS
While GNN-as-Judge demonstrates strong performance, several limitations exist. First, GNN-as- Judge incorporates several hyperparameters related to pseudo-label selection and loss weighting that require selection to achieve optimal performance. For future work, developing automatic methods to select the optimal number of pseudo-labels and reducing hyperparameter dependence could address these limitations. Additionally, updating the model concurrently with pseudo-labe) generation may further improve performance.
I EXTENDED RELATED WORKS
1.1 LARGE LANGUAGE MODELS PREFERENCE ALIGNMENT
Preference alignment refers the process of aligning the outputs of language models with human preferences, often focusing on safety, helpfulness, and factuality (Askell et al., 2021; Ouyang et al., 2022). Reinforcement Learning from Human Feedback (RLHF) (Leike al., 2018; Stiennon et al., 2020) is prevalent method for aligning LMs with human preferences. The RLHF pipeline typically involves collecting human preference data, training reward model, and using reinforcement learning to optimize the language model against this reward function (Bai et al., 2022; Christiano et al., 2017).
While RLHF has proven effective, its reliance on a separate reward model introduces computational and methodological complexities. To address these challenges, Direct Preference Optimization (DPO) (Rafailov et al., 2023) has emerged as a more efficient alternative, climinating the need for an explicit reward model by directly optimizing preference probabilities. Several extensions and refinements of DPO have since been proposed. For instance, KTO (Ethayarajh et al., 2024) incorporates insights from prospect theory to enhance preference learning. Other variants, such GPO (Zhao et al., 2024), VPO (Azar et al., 2024), and ODPO (Amini et al., 2024), further improve or generalize DPO in different theoretical and practical aspects. Recent advancements, including ORPO (Hong al., 2024) and SimPO (Meng et al., 2024), simplify the alignment pipeline by removing the need for a reference model while maintaining competitive performance. Despite these advancements. the application and adaptation of preference alignment techniques to structured tasks such as graph-based learning remain underexplored.
THE USE OF LARGE LANGUAGE MODELS (LLMs)
In accordance with transparency and reproducibility standards, we disclose that LLMs were employed to assist with the preparation of this manuscript. Specifically, we utilized LLMs for language polishing. grammatical correction, and stylistic improvements to enhance the clarity and readability of our technical writing.
28


Reviews:
[{"review_id": "uv1PAu1wMw", "reviewer": "Reviewer_HTs2", "summary": "This paper studies few-shot semi-supervised node classification on text-attributed graphs (TAGs). It proposes GNN-as-Judge, a framework that combines a structure-aware GNN and a text-centric LLM to curate high-quality pseudo-labels and mitigate label noise. The method first selects a subset of unlabeled nodes using an influence-guided criterion, then exploits LLM–GNN agreement and disagreement with GNN preference scores to construct pseudo-labeled data. The LLM is fine-tuned with a weakly supervised objective that integrates instruction tuning on the agreement set and preference tuning on the disagreement set. Experiments on multiple benchmarks show consistent gains, especially under low-resource regimes.", "questions": "n/a", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths:\n1) GNN-as-Judge paradigm: Positioning the GNN as a judge complements the LLM’s lack of structural inductive bias, yielding more reliable pseudo-labels on TAGs.\n2) Collaborative pseudo-labeling: Using both agreement and disagreement—filtered by GNN preference margins—captures easy and hard examples, enhancing learning signals beyond confidence-only self-training.\n3) Weakly supervised fine-tuning: The combined instruction-tuning plus preference-tuning objective leverages informative but potentially noisy hard examples while reducing overfitting risk.\n4) Empirical validation: The approach outperforms strong GNN and LLM-based baselines across 3/5/10-shot settings and shows promising cross-dataset zero-shot transfer, indicating practical value in low-label scenarios.", "weaknesses": "Weaknesses:\n1) Gap Between Theory and Practice: The paper's theoretical proofs rely on simplified assumptions (like linear models and independent errors) that may not hold true in the actual, more complex experimental setup.\n2) Unreliable GNN Judge: The method assumes the GNN is a trustworthy \"judge\" when it disagrees with the LLM. This assumption can fail on graphs with noisy structures or unusual patterns, leading the LLM astray with flawed guidance.\n3) High Computational Cost: The multi-stage pipeline is computationally expensive, demanding significant time and memory. This raises practical concerns about its scalability to massive, real-world graphs.\n4) Complex Hyperparameter Tuning: The framework introduces several new hyperparameters that need careful tuning for each dataset. This adds complexity and makes the method harder to apply effectively without extensive experimentation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0YTfR7cdLO", "reviewer": "Reviewer_EAHZ", "summary": "The core of this paper is to use GNN's ability to understand structures to fine-tune LLM by selecting more reliable pseudo labels. Meanwhile, nodes that do not have particularly reliable labels but show certain preferences are used as soft labels for weakly supervised learning.", "questions": "Q1- Could the author explain in simple terms how the set of disagreement is defined? That is, how is the disagreement set obtained?\n\nQ2- Can the author simplify the algorithm flowchart? I think the entire technical route may not require so many symbols to illustrate.\n\nQ3- What is the citepseer dataset? It seems to be a typo.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1- The author uses a lot of theoretical analysis to prove the rationality of the method.\n\nS2- The experimental results are excellent on multiple datasets.\n\nS3- The technical route is reasonable and there is no technical loophole in the method itself.\n\nS4- This paper is presented at a very high standard of presentation.", "weaknesses": "W1- Researchers have practiced using large language models to label GNN or LLM. The author of this article has elevated the method to a very high concept, namely GNN as a referee, because it can understand structural information, but its essence is still a common process of: 1. select some candidate nodes to query the large model; 2. After the inquiry, further screen for pseudo labels by comparing the predicted results of GNN and LLM; 3. The pseudo labels with high confidence are used as strong supervision signals, while those with relatively low confidence are used as weak supervision signals.  The method in this paper does not involve significant innovation in paradigm, but rather represents a better technological implementation of the three processes mentioned above.\n\nW2- Many of the technical details are referenced but not explained in detail throughout the article. Related work also has the same problem, such as citing the literature but not specifically explaining the core contributions of these methods, resulting in the inability to quickly grasp the core differences between the proposed methods and previous methods after reading.\n\nW3- The paper uses too many symbols to show the content, in fact, I read the process, not careful analysis of definitions, theorems, etc. , can also know what the author's technical route, so some theoretical analysis may be redundant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OyTFhusKmY", "reviewer": "Reviewer_nL51", "summary": "The paper addresses the challenge of generating reliable pseudo labels in a semi-supervised node classification setting where labeled data are scarce. Pseudo-labeling in such cases faces two key issues: (1) selecting which unlabeled nodes should receive labels, and (2) mitigating noise in the generated pseudo labels. To address these, the authors propose an influence-guided node selection strategy, where each unlabeled node’s representation influence from labeled nodes is quantified, and only those with high influence scores are chosen for pseudo labeling. Among these selected nodes, the agreement set nodes where both the GNN and the LLM agree on the label is used as high-confidence pseudo nodes. For the disagreement set, only nodes with a preference score exceeding a predefined threshold are retained. The authors then fine-tune the LLM using a unified objective that combines instruction tuning on the agreement set and preference tuning on the filtered disagreement set. Experimental results on multiple datasets show the proposed method can outperform the recent baselines.", "questions": "1. Theorem 2 assumes independence between GNN and LLM errors. Have you measured the actual correlation or overlap between their error patterns to validate this assumption?\n\n2. Table 5 shows that GNN accuracy in disagreement cases is low. How do you justify treating the GNN as the “teacher” in those instances?\n\n3. Do certain classes benefit more from influence-guided selection or from preference tuning? How does your method perform when the labeled set is imbalanced?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper clearly articulates the challenges of applying LLMs to few-shot semi-supervised learning on text-attributed graphs, particularly the difficulties in generating reliable pseudo-labels and mitigating label noise.\n- The idea of using a GNN as a judge to guide pseudo-labeling for an LLM is conceptually interesting\n- The idea of selecting nodes from the disagreement set is interesting, as these represent challenging (hard) examples where the two models diverge. This is more sophisticated than simply selecting high-confidence (easy) pseudo-labels\n- The influence-guided node selection based on graph structure is well-motivated. Theorem 1 provides a computationally tractable upper bound for node influence which identifies unlabeled nodes most strongly affected by labeled data.\n- Theorem 2 formally justifies why agreement sets yield higher-quality pseudo labels \n\n- The paper includes extensive experimental detail with existing models and datasets, cross dataset experiments, and strong ablation study", "weaknesses": "- The influence metric (Eq. 1) formalizes how labeled nodes affect unlabeled ones through message passing, and high-influence nodes are prioritized for pseudo labeling.  Selecting only high-influence nodes (those strongly affected by labeled data) introduces bias: it oversamples regions near labeled nodes and ignores distant or boundary regions that may be crucial for generalization. The paper provides no analysis of how it affects underrepresented classes. \n\n- The framework's central premise is that GNNs provide more reliable predictions than LLMs in disagreement cases. However, Table 5 shows that GNN predictions in the disagreement set are correct only 31.55% of the time for Pubmed, 37.38% for ogbn-arxiv, and 37.24% for ogbn-products. This creates a logical gap: if the GNN is frequently wrong in disagreement cases (which is why there's disagreement in the first place), then using its predictions to train the LLM could inject substantial noise and potentially degrade performance. Moreover, the paper lacks an analysis to quantify how often: (1) GNN is correct and LLM wrong, (2) LLM is correct and GNN wrong (harmful preference), or (3) both are wrong.\n\n- Additionally, in the preference tuning framework, when GNN and LLM disagree, the LLM's prediction is automatically assigned as the \"dispreferred\" response. However, there is no validation that the LLM is actually wrong in these cases. The dispreferred label could be the correct answer. This creates a scenario where the framework actively penalizes the LLM for making correct predictions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HuTTUaL48i", "reviewer": "Reviewer_19Gq", "summary": "This paper introduces GNN-as-Judge, a novel framework designed to enhance few-shot semi-supervised learning on TAGs by leveraging the complementary strengths of LLMs and GNNs.The key idea is to use the GNN as a structural judge to guide pseudo-label generation and selection for the LLM. The framework first identifies structurally influential unlabeled nodes based on graph topology, then divides them into agreement and disagreement sets according to the consistency between LLM and GNN predictions.Reliable pseudo-labels from the agreement set are used for instruction tuning, while informative but uncertain examples from the disagreement set are used for preference tuning, with GNN confidence guiding label selection.The authors further propose a weakly-supervised fine-tuning objective that integrates both instruction and preference tuning.Extensive experiments on multiple TAG datasets show that GNN-as-Judge consistently outperforms state-of-the-art baselines, particularly in low-resource settings.", "questions": "1.Hyperparameter selection without validation set: In true few-shot scenarios, even validation labels might be scarce. How would you recommend setting K, τ, and λ when validation performance is unreliable?  \n2.Disagreement set contribution: From Table 5, the disagreement accuracy varies dramatically across datasets (31.55% for Pubmed vs. 66.56% for Citeseer). How does this variation affect the final performance? Is there a correlation between disagreement accuracy and overall improvement?  \n3.What happens with different GNN architectures? You use GCN as the default GNN and briefly mention H2GCN for heterophilic graphs. How sensitive is the method to the choice of GNN architecture (e.g., GAT, GraphSAGE, GIN)?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Originality: The paper proposes a novel GNN-as-Judge framework, enabling collaborative pseudo-label generation on text-attributed graphs. This cross-model feedback design is original and bridges two previously distinct paradigms—graph representation learning and language modeling.\nQuality: The methodology is technically sound and well-structured. The framework combines influence-guided node selection, agreement/disagreement-based pseudo-labeling, and weakly-supervised fine-tuning with preference optimization. Each module is motivated, formalized, and validated experimentally, showing consistent improvements over strong baselines under few-shot settings.  \nClarity: The paper is clearly written, with a logical presentation of concepts and a well-organized methodology. Key formulations and algorithmic steps are explained with sufficient detail and supported by intuitive illustrations, making the overall approach accessible to both graph learning and NLP researchers.  \nSignificance: The proposed approach substantially enhances the usability of LLMs for graph-based semi-supervised learning, particularly in low-resource scenarios. By effectively leveraging GNN feedback to improve pseudo-label reliability, the method has strong potential to influence future research on multimodal graph-text learning and weak supervision paradigms.", "weaknesses": "1.No analysis of preference score threshold τ: The sensitivity analysis only shows results for different τ values but doesn't explain why τ=0.7 was chosen or analyze the disagreement set size vs. quality trade-off  \n2.Training time analysis is incomplete: Figure 5 shows total training time but doesn't break down where the time is spent (influence computation, GNN training, LLM inference, LLM fine-tuning)  \n3.No failure case analysis: The paper doesn't discuss scenarios where GNN-as-Judge might fail or perform poorly", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)