Scale-Free Graph-Language Models
Authors: Jianglin Lu
Yixuan Liu
Yitian Zhang
Yun Fu
Venue: ICLR 2025 Poster (2025)
Keywords: scale-free network
language models
graph neural networks
k nearest neighbor graph
Primary area: learning on graphs and other geometries & topologies

Abstract: Graph-language models (GLMs) have demonstrated great potential in graph-based semi-supervised learning. A typical GLM consists of two key stages: graph generation and text embedding, which are usually implemented by inferring a latent graph and finetuning a language model (LM), respectively. However, the former often relies on artificial assumptions about the underlying edge distribution, while the latter requires extensive data annotations. To tackle these challenges, this paper introduces a novel GLM that integrates graph generation and text embedding within a unified framework. Specifically, for graph generation, we leverage an inherent characteristic of real edge distribution—the scale-free property—as a structural prior. We unexpectedly find that this natural property can be effectively approximated by a simple k-nearest neighbor (KNN) graph. For text embedding, we develop a graph-based pseudo-labeler that utilizes scale-free graphs to provide complementary supervision for improved LM finetuning. Extensive experiments on representative datasets validate our findings on the scale-free structural approximation of KNN graphs and demonstrate the effectiveness of integrating graph generation and text embedding with a real structural prior. Our code is available at https://github.com/Jianglin954/SFGL.

Published as a conference paper at ICLR 2025
SCALE-FREE GRAPH-LANGUAGE MODELS
Jianglin Lu Yixuan Liu Yitian Zhang' Yun Fu1.3
*Department of Electrical and Computer Engineering, Northeastem University "Network Science Institute, Northeastern University 3 KKouuy College of Computer Science Northeastern University
ABSTRACT
Graph-language models (GLMs) have demonstrated great potential in graph-based semi-supervised learning. A typical GLM consists of two key stages: graph gen- eration and text embedding. which are usually implemented by inferring a latent graph and finetuning language model (LM), respectively. However, the former often relies on artificial assumptions about the underlying edge distribution, while the latter requires extensive data annotations. To tackle these challenges, this pa- per introduces a novel GLM that integrates graph generation and text embedding within a unified framework. Specifically, for graph generation, we leverage an in- herent characteristic of real edge distribution- the scale-free property- as a struc- tural prior. We unexpectedly find that this natural property can be effectively ap- proximated by a simple k-nearest neighbor (KNN) graph. For text embedding, we develop a graph-based pseudo-labeler that utilizes scale-free graphs to provide complementary supervision for improved LM finetuning. Extensive experiments on representative datasets validate our findings on the scale-free structural approx- imation of KNN graphs and demonstrate the effectiveness of integrating graph generation and text embedding with a real structural prior. Our code is available at https//gtthabbboom///n//////////
INTRODUCTION
Recently, graph-language models (GLMs) have been widely explored in graph-based semi- supervised classification on documents, especially for citation networks (Qin et al., 2023; Yu et al., 2025; Lu et al., 2023; He et al., 2024). When designing a GLM for classification, two key challenges arise: graph generation- how to generate a reasonable graph structure for the given documents, and text embedding- how encode the textual sequences into meaningful semantic features. To ad- dress these problems, various GLMs have been proposed, which can be broadly categorized into latent graph inference (LGI) models and language-assisted graph (LAG) models.
LGI models focus on graph generation and typically rely on feature engineering approaches, such as bag-of-worte (Harris, 1954), TF-IDF (Aizawa, 2003), and skip-gram skip-gram (Mikolov al., 2013), to encode textual sequences into shallow representations. These representations serve as the foundation for optimization objectives that jointly learn the underlying graph structure and node embeddings using graph neural networks (GNNs) (Franceschi et al., 2019; Fatemi et al., 2021; Lu et al., 2023; Kazi et al., 2023). However, LGI models typically rely on artificial assumptions about the underlying edge distribution, which may not accurately reflect the real graph structure.
On the other hand, LAG models assume a predefined graph structure and focus on enhancing text embedding through powerful language models (LMs), such as DeBERTa (He et al., 2021) and GPT (Ouyang et al., 2022). The core idea behind LAG models is to finetune LMs on the target datasets, thereby generating more informative text embeddings (Yu al., 2025; Duan al., 2023; He et al., 2024). However, finetuning a pretrained LM typically requires target datasets with extensive anno- tations (Brown et al., 2020), which is often impractical for semi-supervised learning tasks. When sufficient annotations are unavailable, LM finetuning may become unreliable.
In this paper, we propose a novel scale-free graph-language (SFGL) model that integrates graph generation and text embedding into a unified framework. The proposed SFGL framework addresses
"Corresponding author; jiang] uuout look. com

---

Published as a conference paper at ICLR 2025
the challenges of relying on specious edge distribution assumptions and mitigates the limitations of insufficient supervision in LM finetuning. This is accomplished by identifying an inherent structural nature of real networks for rational graph generation and incorporating graph-based based pseudo-labeler to enhance LMM finetuning. Specifically, we investigate a fundamental characteristic of edge distri- bution in real networks: the scale-free property (Barabási & Bonabeau, 2003; Radicchi et al., 2011; 1; Barabási & Pósfai, 2016), which is primarily characterized by a few highly connected hubs and numerous small nodes with few edges (see Fig. for an example). The scale-free edge distribution is prevalent in real-world networks, particularly in citation networks, making it an ideal structural prior for graph generation. However, the dynamic and sequential nature of its formation process makes efficiently fitting this distribution highly challenging. Fortunately, we reveal that k-neurest neighbor (KNN) gruph, constructed using cosine similarity as the distance metric and an appropri- ately chosen k; closely approximates scale-free network. We subsequently leverage this scale-free graph to develop a graph- besph-based pseudo-labeler, which provides complementary supervision for en- hancing LM finetuning. Consequently, the improved LMs can generate semantically enriched text embeddings. Note that our proposed SFGL model can be trained iteratively to further enhance per- formance. We conduct extensive experiments validate findings and highlight the potential of GLMs built on scale-free structures. Our key contributions are summarized as follows:
We identify two key challenges in existing GLMs: artificial structural assumptions in graph generation and unreliable LM finetuning for text embedding. We propose addressing these challenges simultaneously by exploring a well-grounded graph structural prior.
We leverage the scale-free edge distribution in real networks as our graph structural prior. Our empirical validation and analysis reveal that a KNN graph, constructed using cosine similarity with an appropriately chosen effectively approximates a scale-free network.
To the best of our knowledge, the proposed SFGL is the first work to unify graph generation and text embedding within GLM framework, highlighting the synergistic potential of GNNs and LMs under scale-free structural prior.
2 PRELIMINARIES
2.1 PROBLEM DEFINITION
Given set of documents denoted as D (X2, KX, No,V),, where X2 = {x,}' and Xy {x, ) rep- resent the text sequences of m labeled and unlabeled documents, respectively, and 3/, {y:}* the label set for the labeled samples, with m < n indicating a scarcity of labeled nodes compared to the abundance of unlabeled ones, we tackle graph-based semi- semi-supervised classification task that involves training a model on D to classify unlabeled documents Xy. Here, we face two key prob- lems: graph generation hooe to generate reasonable graph structure for the given documents, and text embedding how to encode the textual sequences into meaningful semantic features.
For graph generation, the main challenge lies in identifying reasonable structural prior that charac- terizes the underlying distribution of edges. The choice of this prior is critical, as it determines both the complexity of the learning model and the inductive bias introduced. We suggest that an effective structural prior should exhibit two key characteristics: real-itt should reflect an inherent attribute of the data; and simple-it should allow for straightforward graph genera- tion without requiring additional model training.
Number nooes dearees
Power Law Distribution
Nodes Hubss
For text embedding, advanced LMs are preferred over tra- ditional feature engineering approaches for their superior ability to capture semantic information from textual se- quences of documents. The primary challenge here lies in improving LM finetuning in a semi-supervised setting with very limited labeled samples. We suggest addressing this challenge with graph-based semi-supervised learning model that propagates supervised information from labeled to unlabeled samples through the graph structure. How- ever, this brings us back to the central question: how to identify a reasonable structural prior for graph generation.
Number degroas (d)
Figure 1: Musstrtioo of a scale-free network, highlighting a few hubs and large number of small nodes.
2

---

Published as a conference paper at ICLR 2025
(a) l =
(b) 3
(c) k=
(d) & = 6
Figure 2: Uniform node distribution within structured arrangements causes consistent connectivity.
2.2 SCALE-FREE NETWORKS
We investigate a fundamental structural nature of real-world networks: the scale-free property (Barabási Albert, 1999; Barabási Bonabeau, 2003). In essence, real citation networks ex- hibit scale-free characteristics (Redner, 1998; Hummon & Dereian, 1989; Radicchi et al., 2011) 1). Their degree distribution P(0) follows power law: P(0) x A-a, where denotes the degree of node and is a scaling parameter of the distribution (Clauset al., 2009). For illustration, Fig. depicts the degree distribution of a scale-free network with a = 2.8. As observed, the distribution is heavy-tailed, with few highly connected nodes, known as hubs, and many small nodes with only a few edges (Barabási & Albert, 1999; Barabási & Pósfai, 2016). We suggest that leveraging the scale-free property for graph generation is strong choice, as it reflects an inherent characteristic of citation networks and aligns with our real principle. The challenge lies in constructing a latent graph that can (approximately) form a scale-free network.
2.3 K-NEAREST NEIGHBOR GRAPHS
We begin by exploring KNN graphs, as their construction aligns with our simple principle. The construction process is straightforward: for each node, identify its k-nearest neighbors using specified distance metric based on node features and establish an edge between the node and each of its & nearest neighbors. The degree distribution of KNN graphs is primarily shaped by three key factors: ① node distribution, which is unknown a priori; @ the choice of similarity metric, which affects the probability of node connections; and 3 the value of k. which directly determines node degrees. Without detailed knowledge on these factors, characterizing the degree distribution of KNN graphs remains challenging. We first explore simple cases and analyze each factor independently.
In the absence of prior knowledge about the underlying node distribution, we analyze attribute- free nodes that are uniformly distributed across regular grid structures, including circles, hexagons, quadrilaterals, and triangles. As shown in Fig. 2, the subgraph formed by any node and its first-order neighbors exhibits characteristics of a random network (Barabási & Albert, 1999). Within specified value of kt the connection probabilities between Thus, building KNN graphs on such uniform distributions yields consistent node connectivity.
nodes within the subgraph remain identical.
In KNN graphs, various distance metrics can be used measure similarity between nodes, including Euclidean, Manhattan, and Cosine distances. Fuclidean distance sensitive to variations in feature scale and highly influenced by outliers. Manhattan distance is more robust to outliers but disregards the actual geometric relationships between data points. Cosine distance is particularly suitable for high- nigh-dimensional, sparse data, it prioritizes vector direction over magnitude.
The value of k directly determines the magnitude of node degrees. As increases, more edges are formed, leading to higher node degrees. In the extreme case where k m + the KNN graph becomes a complete graph, where every node is connected to all others and has the same degree.
SCALE-FREE GRAPH-LANGUAGEE MODEL
3.1 SCALE-FREE GRAPH GENERATION
Hypothesis & Analysis. In this paper, we argue that constructing a KNN graph that approximates scale-free network is practically feasible. To support this, we first present the following hypothesis:
3

---

Published as a conference paper at ICLR 2025
A k-nearest neighbor graph, when constructed on citation networks using cosine similarity as the distance metric and an appropriately chosen k. approximates a scale-free network.
To investigate the proposed hypothesis, we present the following definitions and propositions. Definition 1. Given an undirected graph G := (V,E) with |V] nodes and [E] edges, the degree centrality of node u is Cp(c)= = 8(v), where ((v) is the degree of v.
Definition 2. Given directed graph 9 == (V, with [V] nodes and |8| edges, the in-degree centrality and out-degree centrality of node v are CID(V) 0.(v) and CoD(v) = =o(v), where (c(v) and (t(v) are the in-degree and out-degree of respectively.
Proposition 1. For a scale-free network, assuming that there is at most one node v whose degree belongs to 1mmnn 20), we have Iman = mmin I where O mmo and mmin refer to the maximum and minimal degree in the scale-free network.
Proposition 2. For a KNN graph, we have ΣvEy Cιd(T) = Evev 2 _ - k. where [8] S k.V.
=
Considering their inherent biases, KNN graphs exhibit significant similarities to scale-free networks, which we detail as follows. ! Homophily: citation networks exhibit a high degree of homophily, where connected nodes usually have high probabil- ity of belonging to the same class. Notably, the KNN the homophily assumption
algorithm is also based that similar nodes exist in close proximity. @ Directed: citation networks are inherently directed since citations have a clear direction from the citing publication to the cited publication. Coincidentally, the KNN graph is
10-2
Probelaility ptd)
30
10"4
also directed as node being in the k-nearest neigh- bors of node u does not imply node u in the k-- nearest neighbors of node 3 Upper bound of degree: the size of hubs in both scale-free networks and KNN graphs is constrained. Proposition I indicates that the size of hubs in scale-free networks has an upper bound of dmin M Note that the maximal in in-degree in KNN graph is also bounded, and the out-degree of each node is always fixed as k as shown in proposition 2.
Probability Density
Power-Law Exponential
10c
10
In Ingrege (d)
Figure 3: Curve fitting to the in-degree distribution of a KNN graph (k = 5) using Fuclidean distance as the metric.
We discuss the underlying reasons and rationale for using "cosine similarity" and the term "ap- proximates". Cosine similarity focuses on the orientation of feature vectors rather than their mag- nitude, aligning with the syntactic patterns of word semantics. Consequently, few hubs with high in-degrees will appear in KNN graphs if the feature space exhibits some common orientations. The term "approximates" is used because KNN graphs are constructed based on node features, which influenced by the feature engineering approaches employed. After feature extraction, the node distribution may not accurately represent the original data distribution of citation networks.
Empirical Validation. To validate our hypothesis, we construct KNN graphs using cosine similarity with k: = 5 and examine their degree distribution across multiple citation networks, including Cora, Pubmed, and ogbn-arxiv (see Appendix A. for details). Figure 4 presents the in-degree dis- tribution of KNN graphs on these datasets. Our observations reveal that the generated KNN graphs contain massive small nodes alongside few hubs, resulting in in-degree distributions that resemble those of scale-free networks as depicted in Fig. 1. Notably, the degree distributions exhibit slight variations across datasets, which can be attributed to differences in node distributions and the feature extraction methods used to construct feature vectors. To further investigate these degree distribu- tions, we fit the in-degree distributions to exponential and power-law models-bott both characterized by heavy-tailes behavior. Figure 5 illustrates the fitting curves on log-log axes using logarithmi- cally spaced bins. As shown, the in-degree distributions of KNN graphs align more closely with power law, with estimated scaling exponents (a) of 3.3. 2.8, and 2.7 for Cora, Pubmed, and ogbn-arxiv, respectively. These empirical findings suggest that KNN graphs effectively capture the structural properties of citation networks, thus supporting our hypothesis.
We further investigate the degree distribution of KNN graphs using Euclidean distance and present the fitting curves in Fig. 3. As shown, the resulting degree distribution does not fit well to a power-
4

---

Published as a conference paper at ICLR 2025
.00
Carve (ieDegruu Listribution
Gurve inDDggus Destribution
Curve of InsDegras Uncriiution
Kasimal Hub
Maximat Hub
Mastmall Hub
2.73
Hulp
Hubs
Huls
250
Numberr of Modes
Number Noxies
6000
Number of Modies
125
5000
100
4000
400
3000
2000
200
Number (d)
Number
Kumber deg
(a) Cora
(b) Pubmed
(c) wownnaaaaee arxiv
Figure 4: In-degree distribution of KNN graphs on different datasets, where the top-5 largest hubs
marked in red.
12"4
Probabitty pid)
Probatityy pid)
12-1
12-4
10
22-4
Pibbbility Denstyy
10"1
Procability Dersity
10-9
Probacility Densky
12-11
Exponential
RosserLLaw
10't 10
Power-aww
Exconential
10-11
Roseen-Jaw
Exploneniial
In-Degree
Ingegeee
(a) Cora
(b) Pubmed
(c) ogbn-arxiv
Figure Curve fitting to the in-degree distribution of KNN graphs on different datasets.
law model, indicating that Fuclidean distances between vector points fail to accurately capture the similarities between textual features in citation networks. This is because Fuclidean distance pri- marily emphasizes the magnitude of vectors in linear space, neglecting important elements such as the angular distribution or relative orientation of vectors, which are essential for capturing syntactic patterns and semantic relationships of textual attributes. More comparisons are provided in Sec. 4.3.
Graph Construction. Building on the above analysis and validation, we generate a scale-free KNN graph for the given documents D. We begin by encoding the text attributes X' {& U Xu) [x:] mmtn into hand-crafted feature representations using a text encoder Enc(-): f, = Enc(x;), where f; represents the extracted feature vector for x;, and Enc(.) can be either traditional model or a language model. Next, we construct KNN graph on features F using cosine distance: A KNNcos (F, k), where F [fi, fq,..., J. A the adjacency matrix of the resulting graph, is the number of nearest neighbors, and KNNcos() (-) denotes the KNN algorithm with cosine distance. For simplicity, we consider a binary graph, i.e., Aij 1 if Ais topk(A Aij otherwise.
3.2 IMPROVED TEXT EMBEDDING
The previous section addresses the challenge in graph generation, outlined in Sec. 2.1. This section focuses on the challenge in text embedding, i.e., improving the finetuning of LMs.
Leveraging the graphs generated in Sec. 3.1, we design graph- based semi- supervised learning model to produce high-quality pseudo-labels, which enhance LM training. Given the node features F, adjacency graph A, and available labels Y/ m train GNN using the standard node classification loss:
Pseudo-Labeler.
LUKN Σ y In Zi, 2, Softmax(GNN (f:, A)),
where & represents the GNN parameters. Here, any GNN model, such graph convolutional networks (GCN) (Kipf& & Welling, 2016), GraphSAGE (Hamilton et al., 2017), or graph attention networks (GAT) (Velickovic et al., 2017). be employed (see Sec. 4.3 for experimental compar- isons). Once trained, the pseudo labels y, for all unlabeled nodes (j E (I.....n]) are generated by yj arggaa(GNNoo))).. (fy, A)), enabling effective label propagation.
5

---

Published as a conference paper at ICLR 2025
Text Embedding. To extract semantic-enriched text embeddings from documents, finetune LMs on D Nu, De, where Di- {ý;}", " using the following loss function:
1 m im1
yi las +E s Insj.
Lim
where S; = softmax((MM)) x:)) and LM (.) pretrained LMM with parameters θ After finetuning, for each node X,, i 21, m + n). we generate its text embedding e, by extracting the last hidden states of the finetuned LM, i.c., e; = (M)...).
Following (He et al., 2024), also utilize the large language model (LLM) GPT3.5 (Ouyang al., 2022) to enhance text embeddings. We access GPT3.5 through its open API and treat it as an oracle, feeding it prompt imput p, and obtaining the corresponding textual response r, for each node i:
r: LLM():), P: = concat q),
(3)
where the prompt p: is concatenated by the text sequences x; and a task-specific question q (He al., 2024). The question q is standardized and universal for all nodes, which prompts GPT3.5 to classify each node and provide an explanation for its prediction (see Appendix for more details). Note that, do need to finetune GPT3.5 or access its inner structures or embeddings. Once the textual response r; is obtained for x,, we replace x, with and finetune the LMe(-) (-) to generate updated text embeddings for the documents.
3.3 CLASSIFICATION
Up to this point, we have addressed the two key problems outlined in Sec. 2.1, obtaining semantic- enriched text embeddings E [ei, e,,..., em-nn and meaningful graph A. We can now tackle the classification task by training graph-based semi-supervised learning model on E and A using the available labels Vz. Following (He et al., 2024), we utilize a GNN as the classification model. We also evaluate the performance of the LMs with generated pscudo-labels. Notably, the proposed SFGL can use an iterative training mechanism. The improved text embeddings facilitate the gen- eration of higher-quality pseudo-labels, which, in turn, further enhance the discriminability of the embeddings. We explore this iterative training strategy in Sec. 4.3. Due to space limitations, the detailed training stages are summarized in Appendix A.5.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Datasets. We conduct extensive experiments on four citation networks: Cora (Sen et al., 2008), Pubmed (Kipf& & Welling, 2016), ogbn- (Hu et al., 2020), and arxiv23 (He et al., 2024) (see Appendix A. for more details.). Following standard practice, we utilize traditional text en- coders to process textual attributes for graph generation. For Cora, each publication is represented using a bag-of-wordz model (Harris, 1954) with a vocabulary of 1, 433 unique words. For Pubmed, documents are encoded as TF/DF weighted word vectors (Aizawa, 2003) from dictionary of 500 unique words. For ogbn-arx: model (Mikolov et al., 2013) and word2vec model (Mikolov et al., 2013), respectively. Additional comparisons with other text encoders are provided in Sec. 4.3. In our experiments, the learning models have access node attributes, while edge attributes remain unavailable.
and arxiv23 v23, textual features are extracted using a skip-gram
Baselines. We compare our approach against several state-of-the-art methods. @ GCN (Kipf & Welling, 2016): trains GCN using hand-crafted shallow features {f.] mm and labels {yr) "m
@ DeBERTa (He et al., 2021): finetunes pretrained DeBERTa on raw text attributes { and labels m @ GCN+DeBERTa (Zhao al., 2022): trains a GCN using text embeddings [m and labels (y:)m.. where [c-mmt are obtained by finetuning DeBERTa on y.)".. @ DeBERTa+GPT (Ouyang et al., 2022): finetunes a DeBERTa using GPT-gneerated responses {r,) of labeled samples and labels {y.}m. @ GCN-DeBERTa+eRR GPT (He et al., 2024): trains GCN using text embeddings me and labels (y.)". where [c]]** are obtained by finetun- ing DeBERTa on {r,.,, mm @ SFGL (ours): trains GCN using text embeddings {e, 1m+ and labels (y.)",, where {e, [e.]mt are obtained by finetuning a DeBERTa with pseudo-labes labels {Y,)". +
SFGL+GPT T (ours): similar SFGL but uses GP -egeerereteed responses to finetune the DeBERTa.

---

Published as a conference paper at ICLR 2025
Table 1: Performance of different models trained four datasets with varying labeling rates, where indicates unreliable results due to an extremely limited number of training labels.
Datasets & Method
Labeling Rates (labels)
Cora
(708
5.17%)(140)
7.75%((210
15.51%((20)
22.16%((o00
GCN
0.5898 0139
0.6038
0.635810.0164
0.6936 0055
0.711440.0m 0040
DeBERTa
0.2406 0799
0.304810.0118
1.366811.0683
0.6016 0495
0.6904 041013
GON+DeBERTA
0.2810=a.0136
0.3598 0000
0.4744+t0.120 12181
0.6482 02.8
0. 2..99000.0280
SFGL (curs)
0.6086-4.01440
0.631440.0191
0.6612+0.02a0
0.6942-a..100
0. 0.720440.0133
DcBERTa+CPT
0.2776=a.0122
0.5450
0.662440.o0dd
0,6888
0.7082
GON+DeBERTa+GP
0.4862 0489
0.6272:1.0211
0.6550
0.6836=00 01993
0.697810.cm 0085
SFGL++GPT (curs)
0.6348 01336
0.685610.0131
0.6992
0.7278 20.0781000041
0.737420.c072
Pubmed
0.30%
0.61%((120)
0.91%)(150)
L83%)(360)
2.545%(500)
GCN
0.6770=0.0217
0.702220.0107
0.7398
0.738820.000
0. 0.746420.0100
DeBERTa
0.3616
0.4114
0.4402210.057
0.5732=0.008
20.033
0. 0.747440.083
GON+DeBERTa
0.4772++0.0112
4.5046+0.0180
0.5520 +-000477
0.6648
066-4840.0453.
0.8042 +700777
SFGL(curs)
0.7238.40.0124
0.7792 02226
0.8188
+.00090
0.8524-40.ogg
0.0000
0.8390 00,204
DeBERTa+CPT
0.451440.070r
0.5206 +0. 1020
0.6196 1..0..440
0.9278
#000044
0.9274 u0.
GCN+DCBERTa+GPT
0.7386 0129
0.7440+0.om 0272
0.7868
0.9242
0049
0.9282:0.om 0022
SFGL+GPT (curs)
0.86 038
0.901 1.901410.0064
0.9088 1000037
0.9162 00009
0.9116
ogbn-arxiv
0.03%
0.06%;(100)
0.09%
0.18%)(300)
0.30%)(500)
GCN
0.3375
0.357840.0179
0.3991
0.4227
0,451620.008
DeBERIa
0.2541 06606
GON+DeBERTa
0.1661=0.0544
0..
0.2130=a.01ep 0.1964
0.235920.0244
(HGI.
1.325110.03m
0.4171 20.0775
1..037
0.4979 2:0 0238
0.5577 1000060
DeBERTaB*PP
0.1653=2..0m
0.2892
GON+DeBERTa+GGY
0.2137 40.0317
0.2200
0.2188 4.00709
0.2321
0929
0.26444..0018
SFGL4GPT (curs)
0.4570 0625
20.00720.0172
0.6380
0.636120.014
0. 7..5677000182
arxiv23
0.11%
0.22%)(100)
0.32%
0.65%(300)
1.08%)(500)
GCN
0.3809
0.4229
0.4342
0.4771
0.4976
DeBERTa
0.2204 40.0444
0.2972
20.0399
0.5619#u.0m
GON+DeBERTa
0.2506=a.0140
0.2659#0.01ue
0.2648 #0.0067
0.2803
0.2976
SFGL(curs)
0.4323 0259
0.500210.0125
0.5160 10.0117
0.5699
0036
0.5973
0080 10.000
DeBERTa+GPT
0.2223 1..22310.0158
0.3967 0.10777
0.6137 1:0 0497
GON+DeBERTa+GR
0.3261 10 0389
0.317440.0400
0.3851 1000113
0.397 0307
0.4686 0199
SFGL+GPT (curs)
0.4953 0066
0.5811 10.0167
0.604140.0164 +0.0164
0.6439_a..133
0.65522n.0245
4.2 COMPARISON ANALYSIS
Table I presents the classification performance of various methods across different datasets and la- beling rates. Our key observations are as follows: ① Across all datasets, classification accuracy generally improves the labeling rate increases. This aligns with expectations, more labeled data provides better supervision, enhancing model performance. @ The baselines DeBERTa and DeBERTa+GPT perform poorly low labeling rates, particularly ogbn-arxiv and arxiv23, where finetuning LMs becomes unreliable under extreme label scarcity. This is because insufficient labeled data fails to provide sufficient information for gradient updates, leading to the learning of overly narrow patterns, which impairs generalization. However, incorporating additional pseudo- labels enables LMs to generate informative text embeddings, thereby enhancing performance. 3 Incorporating GPT-generated responses significantly enhances both GCN and DeBERTa, particu- larly on the Pubmed dataset. This highlights the potential of L.LMs extract and leverage rich semantic information for text modeling. @ In most cases, the proposed SFGL and SPG achieve the highest performance, demonstrating their effectiveness in luw-supervision scenarios.
4.3 ABLATION STUDY
Iterative Performance. As mentioned in Sec. 3.3, improving the GNNs allows us to generate more accurate pseudo-labels. These enhanced pseudo-labels, in turn, enable us retrain the LMs, pro-
7

---

Published as a conference paper at ICLR 2025
Table 2: Performance comparison of the proposed method with and without iterative training.
Labeling Rates (labels)
(o30%
0.61%
0.91%(188)
1.83%(3300)
2.54%
SFGL
0. 2..22
0.7792
0000263
0.8188
01.0
0.839044..zae 4..2200
SPGL+GCN
0. 0..434
0.
0.8686=0.oom
0.8976
0.896644.uud
SFGL+GCN
0.891 2 0293
0.8974 0.0116
0.9110 =0. 0083
0.9066
0.918610 0073
SFGL+GPT
0.86/10=...ae
0.90144+0.0064
0.9088 1088800.0000 000077
0.9162
0.91 1001660.0027 0027
SFGL+GPI+GCN
0.9142 =0014200000800 D0.00
0.9250+b.oodd
0.9294=0.oood
0.9290
0,9374 000011
SFGL+GPI+GCN
0.9138 0.9198
0.9216
0.9240
0.9264
Table 3: Performance comparison of different models using various distance metrics, including Manhattan (M), Euclidean (E), and Cosine (C) distances.
Labeling Rates (labels)
2.58%((70
5.17%((140)
7.75%(210)
15.51%((22)
22.16% 1,0
SEGL (M)
0.5528 0.5454_0.0027
0.6206go.0101
0.6810
0. 0.714410.0109
SPGL (B)
0.5178.0.0271
0.5524=0.160s
0.6240
0.6718#0.0r
0.7138
SFGL (C)
0.6086
0.6314_0.0191
0.6612+0.02m0
0.6942 0100
0.7204
SPGL+GPT(MM
0.6176=a..099
0.6564
0.666640.00229
0.6954 0215
0.7214
SFGL+GPT (2)
0.6138
0.647420.0152
0.6588+..0010
0.6932 0249
0. 70.08800000
(C)
0.634820.0136
0.685620.0131
0.6992
0.7278 0041
0.737440.0072
DeBERTa+GCN (M)
0.4956-a.v136
0.5286-0.1312
0,613440..0o1
0.6922
0. 0.7282 0.7282+0.0000 0000000
DeBERTa+GCN (E)
0.1660=0.028
0.512620.0162
0.619110.0100
0.6882 0087
0.725444..0m
DeBeRTa+GCN (C)
0.6232 0094
0.6616_0.0077
0.6728
0.706 0077
1..21140000
' [M]
0.3388
0.6780..o.0034
0.665611.01122
7.70922 0088
70.7332
10.0109
DeBERTa+G2I+GON (E)
0.5268=a.0146
0.6412=0.0110
0.7112 0,6634+0.0077
0.7318
0.7318 00.0110
DeBENTa+GPT+GCNTCC
0.6588_o.0113 0.6978_0.0200
0.6962+0.0079
0. 0.7206go.010p
0.7426 0.7426=2.0077 00.0072
Table 4: Performance compaarion between real graphs and KNN graphs.
Labeling Rates (labels)
(.11%
0.22%((00)
0.32%((50))
(.65%(300)
1. 1.08%(300)
Real
Graph
0.3691 10.0033
0.4124
1412440.0112 0185
0.4354
0.4764
0.5039 10.0071 0001
KNN
Graph k-10
0.3660 01388
0.4031 :: 0.0054
0.4152 0145
0.4541.1.011
0.4706 0164
Graph k-20
0.380944..025)
0.4229
0.4342 000002
0.4771 20.007
0.4976
KNN Graph k=30
0.3795
0.4085 0.002
0.4130 000009
0.45955a..014
0.4864 0007
XNN Graph k=50
0.38 1.381822000144
0.408 = 0081 0.008
0.4420320.0133
0.4679
0.4857 200007
KNN
Graph k=10c
0.3767#4.00s0
0.4100
0.4201 40.0091
0.4778
0.5005
ducing more refined text embeddings (see Appendix A.5 for detailed training stages). To evaluate this iterative training strategy, we introduce two improved GNN variants, GCN and CCN, derived from SFGL and SF GL++TP,, respectively, new pseudo-labelers. Table 2 presents the iterative performance of our models on the Pubmed dataset. Our results demonstrate that this iterative mech- anism brings further performance improvements, highlighting the mutually beneficial relationship between GNNs and LMs in iterative training. Notably, SFGL+GPT+ GCCN consistently outperforms SFGL+ I + GCN, indicating slight overfitting during the iterative training process.
Different Distance Metries. As discussed in Sec. 2.3, various distance metrics, such as Euclidean, Manhattan, and Cosine distances, can be used KNN graphs measure similarity between sam- ples. To evaluate their impact, we compare model performance using these different metrics. Table 3 presents the results for SFGL, SFGL+GPT, D and DeBERTa+GPT-CC two models use GCNs as the final classifier, while the latter two employ LMS as the classifier but leverage our graph-based pseudo-labeler to enhance LM finetuning. As shown, Cosine distance con- sistently outperforms the other metrics, particularly at very low labeling rates, further justifying its effectiveness for graph construction in our framework.
The first
Real Graph vs. KNN Graph. Table 4 presents the performance comparison between real graphs and KNN graphs on the arxiv23 dataset, considering different labeling rates and values of k. No- tably, under extremely low labeling rates, the performance achieved with the true graph is not sig- nificantly better- and in some cases, even worse- worse-thann that obtained with a pre-constructed KNN graph. This suggests that true graph may not be crucial for improving LM finetuning, particularly
8

---

Published as a conference paper at ICLR 2025
in scenarios with very limited supervision. Our explanation for this phenomenon is that both the number of labeled nodes and the choice of which nodes to label play a crucial role in determining the final classification performance. Consider an extreme case: if all labeled nodes in real graph are isolated, label information cannot propagate to the unlabeled nodes. In contrast, constructing an artificial graph for these nodes may establish connections between the unlabeled nodes and the iso- lated labeled nodes, facilitating label propagation across the graph. Consequently, artificial graphs can potentially outperform real graphs. particularly in scenarios with very limited supervision. See Appendix A.4 for additional experimental results.
5 RELATED WORK
Latent Graph Inference Models. Traditional LGI methods adopt linear projections to learn node representations and optimize various objective functions to learn latent graphs. For example, Zhang et al. (2010) design an entropy regularization to control the uniformity level of edge weights. Nie et al. (2016) infer an optimal graph by assigning adaptive neighbors. Lu et al. (2018) impose spec- tral sparsity on the graph Laplacian matrix. Lu et al. (2021a) assume that two samples are likely to belong to the same class if one sample is close to the reconstructed representation of the other. Ad- vanced LGI models exploit GNNs to learn the latent graphs. For instance, Franceschi et al. (2019) model a discrete probability distribution for the edges. Fatemi et al. (2021) provide supplementary supervision for latent graphs through selff-uuerviiiion task. Lu et al. (2023) propose model- agnostic model that obtains supplementary supervision directly from true labels. However, existing methods typically rely on artificial assumptions about the underlying edge distribution. For example, Zhang et al. (2010) impose a uniformity assumption on edge weights. Lu et al. (2018) introduce a block diagonal prior to the graph Laplacian matrix. Lu et al. (2021b) construct an adaptive neighbor- hood graph by assuming the probability property of edge weights. Fatemi et al. (2021) assume that a graph structure effective for predicting features is also effective for label prediction. These artificial assumptions may not accurately reflect real graph structures and require specific optimizations with additional model training across the entire dataset.
Language-Assisted Graph Models. LGI models typically rely on feature engineering approaches, such as skip-gram and TF-IDF, to encode textual sequences into feature vectors. In contrast, re- cent LAG models seek to enhance text embeddings by leveraging various LMs extract richer semantic features from text sequences (Li et al., 2024). For example, Zhao et al. (2022) design a variational expectation- maaiiiizaiaiiiiiiiiiiiomm framework to fuse graph structure and language learning for classification. Duan et al. (2023) first conduct parameter-efficient finetuning on a pretrained LM and then generate text embeddings using the finetuned L.M. He et al. (2024) leverages LLM to capture textual information and applies a small LM as the interpreter to transform the responses of the LLM into informative features. Yu et al. (2025) use LLMs generate nodes and edges for text- attributed graphs, which harnesses LLMs for enhancing class-level information. For most existing LAG models, finetuming an LM on the target dataset is essential to generate semantically enriched text embeddings. However, it is notorious that finetuning a pretrained LM typically demands large amount of annotated data (Brown al., 2020), which poses significant challenge for LAG models in cemissupervised learning tasks, where available annotated data is often scarce.
6 CONCLUSION
In this paper, we identify two primary challenges in exiting graph-language models for semi- supervised learning, i.e., artificial structural assumptions in graph generation and unreliable LM finetuning for text embedding. We tackle these challenges by establishing well-grounded struc- tural prior. Specifically, we examine the scale-free property of citation networks and reveal that this structural characteristic can be effectively approximated using simple KNN graph. Building on this observation, we propose a novel graph-language model that employs a customized KNN algo- rithm for scale-free graph generation and utilizes wraph-based pseudo-labets to provide additional supervision for improved text embedding, Extensive experiments on representative datasets validate our findings on the scale-free structural approximation of KNN graphs and demonstrate the effec- tiveness of integrating graph generation and text embedding with real structural prior. We hope this study highlights the synergistic potential of GNNs and LMs, providing valuable insights for researchers in both the GNN and NLP communities.

---

Published as a conference paper at ICLR 2025
REFERENCES
Akiko Aizawa. An ciemmatttiio-eooetiie perspective of tf-idf measures. Information Processing & Management, 39(1):45 1):456555, 65, 2003.
Albert-Láászóó Barabási and Réka Albert. Emergence of scaling in random networks. Science, 286 (5439):509-512, 1999.
Albert-László Barabási and Eric Bonabeau. Scale-free networks. Scientific American, 288(5):60-69, 2003.
Albert-LászIó Barabási and Márton Pósfai. Network science. Cambridge University Press, Cam- bridge, 2016. ISBN 9781 107076266 1 1070776269.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Vons Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz. Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., 2020.
Aaron Clauset, Cosma Rohilla Shalizi, and Mark EJ Newman, Power-law distributions in empirical data. SIAM review, 51 (4):661-703, 2009.
Keyu Duan, Qian Liu, Tat-Seng Chua, Shuicheng Yan, Wei Tsang Ooi, Qizhe Xie, and Junxian He. Simteg: A frustratingly simple approach improves textual graph learning, arXiv, abs/2308.02555 2023.
Bahare Fatemi, Layla El Asri, and Seyed Mehran Kazemi. Slaps: Self-supervision improves struc- ture learning for graph neural networks. In Advances in Neural Information Processing Systems, pp. 22667-22681, 2021.
Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. Learning discrete structures for graph neural networks. In Proceedings of the 36th International Conference on Machine Learning, pp. 1972-1982, 2019.
Will Hamilton, Zhitao Ying, and Jure Leskovee. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Zellig S Harris. Distributional structure. Word, 10(2-)):146-66. 1954.
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. In International Conference on Learning Representations, 2021.
Xiaoxin Hc, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Har- nessing explanations: Lim-to-Im interpreter for enhanced text-attribuedd graph representation learning. International Conference on Learning Representations, 2024.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22 18-22133, 2020.
Norman P Hummon and Patrick Dereian. Connectivity citation network: The development of dna theory. Social networks, ((1):39-633, 1989.
Anees Kazi, Luca Cosmo, Seyed-Ahmad Ahmadi, Nassir Navab, and Michael M. Bronstein. Dif- ferentiable graph module (dgm) for graph convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4562):666-1617, 2023.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In International Conference Learning Representations, 2016.
10

---

Published as a conference paper at ICLR 2025
Chaofan Li, Minghao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. Making text embedders few-shot learners. In The Thirteenth International Conference on Learning Representations, 2025.
Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. A survey of graph meets large language model: Progress and future directions, 2024. URL https:
//arxiv. org/ abs 22112211.101 12399.
Canyi Lu, Jiashi Feng, Zhouchen Lin, Tao Mei, and Shuicheng Yan. Subspace clustering by block diagonal representation. IEEE transactions on pattern analysis and machine intelligence, 41()): 487-501, 2018.
Jianglin Lu, Jingxu Lin, Zhihui Lai, Hailing Wang, and Jie Zhou. Target redirected regression with dynamic neighborhood structure. Information Sciences, 544:564-588 2021a.
Jianglin Lu, Hailing Wang, Jie Zhou, Yudong Chen, Zhihui Lai, and Qinghua Hu. Low-rank adaptive graph embedding for unsupervised feature extraction. Pattern Recognition, 113:107758, 2021b. ISSN 0031-3203.
Jianglin Lu, Yi Xu, Huan Wang, Yue Bai, and Yun Fu. Latent graph inference with limited supervi- sion. In Advances in Neural Information Processing Systems, 2023.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representa- tions of words and phrases and their compositionality. Advances in neural information processing systems, 26, 2013.
Feiping Nie, Wei Zhu, and Xuelong Li. Unsupervised feature selection with structured graph opti- mization. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744 2022.
Yijian Qin, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Disentangled representation learning with large language models for text-attributed graphs. arXiv, abs/2310 18.3110..1..,.. 2023.
Filippo Radicchi, Santo Fortunato, and Alessandro Vespignani. Citation networks. Models of science dynamics: Encounters between complexity theory and information sciences, pp. 233-257, 2011.
S. Redner. How popular is your paper? an empirical study of the citation distribution. The European Physical Journal B, 4(2):131-134, August 1998. ISSN 1434-6028.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. Al Magazine, 29(3):93, 2008
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Ben- gio, et al. Graph attention networks. stat, 10-45500 2017.
Liang Wang. Nan Yang. Xiaolong Huang, Linjun Yang. Rangan Majumder, and Furu Wei. Improv- ing text embeddings with large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 11897-11916, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
Jianxiang Yu, Yuxiang Ren. Chenghua Gong, Jiaqi Tan, Xiang Li, and Xuecang Zhang. Leveraging large language models for node generation in few-shot learning on text-attributed graphs. In Proceedings of the AAAI conference on artificial intelligence, 2025.
I Limei Zhang, Lishan Qiao, and Songcan Chen. Graph-optimized locality preserving projections.
Pattern Recognition, 43(6)::993-2003 2010.
Jianan Zhao, Meng Qu, Chaozhuo Li. Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. Lear- ing on large-scale text-attributed graphs via variational inference. In The Eleventh International Conference Learning Representations, 2022.
11

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 DATASET DESCRIPTION
We conducted extensive experiments on four text-attributed graph benchmarks: Cora (Sen et al., 2008), PubMed (Kipf& & Welling, 2016), ogbn-arxiv (Hu al., 2020), and arxiv23 (He al., 2024). The Cora dataset contains 2, 708 papers belonging to seven different classes. The PubMed dataset includes 19, 717 publications categorized into three classes. The ogbn-arxiv (Hu et al., 2020) dataset includes 169, 343 arXiv papers across 40 subject areas in computer science. Labels for this dataset are manually annotated by the authors and arXiv. The arxiv23 (He et al., 2024) dataset includes all computer science papers submitted in 2023 or later on arXiv. Similar to ogbn- arxiv, these papers are divided into 40 classes.
A.2 MODEL. CONFIGURATION
We follow (He et al., 2024) for model configurations. For GNNs, we use a two-layer GCN (see Sec. 4.3 for comparisons with other GNN architectures). The hyper-parameters for hidden dimension, learning rate, dropout ratio, and weight decay are set to 128. 0.001, 0.5, and 0.0005, respectively. For LMs, we finetune a pretrained DeBERTa (He al., 2021) on the target datasets. The batch size, learning rate, and dropout ratio are set to 20, 2 x 10-5, and 0.3, respectively. We also em- ploy GPT3.5 (Ouyang al., 2022) for inference. We empirically set k to 25, 15, 25, 20 for Cora, Pubmed, ogbn aaxiv,, and arxiv23, respectively. We conduct five independent experiments with different random seeds and report the average test accuracy along with the standard deviation.
For LLM prompt inputs, we follow established methods (He et al., 2024). Each prompt consists of paper's title and abstract, along with a task-specific question designed to elicit class prediction and corresponding explanation from the LLM. The generated textual responses by the LLMs serve as inputs for the LMs, which use them to generate text embeddings for all nodes. These embeddings then further processed by GNNs to enhance model performance.
A.3 PROOF OF PROPOSITION
Proposition 1. For a scale-free network, assuming that there is at most one node v whose degree belongs to 1mmax 20), we have maax = 6min [V] --,, where O mma and O 0min refer to the maximum and minimal degree in the scale-free network.
Proof. We write the degree distribution PP(0) of a scale free network as: P(0) = =0000-0--- 50-0 where is the degree of node, & is constant, and is scaling parameter of the distribution. Since
d I-a + constant and Jem... P()) 1, we have ? - 0- 00 I-a 0ms.
02-a
As result, we obtain (a > 1), we have - Amin a-1 (a - 1)0m=1 and PPO)) (a 1)0mm 0~a Assume that there is most one node whose degree belongs (0mann 00), where Omar is the maximum degree in the network. We have Jo... P(0)dd
Since lom I-a
g1-0
1 d0 = [ simplification, we obtain (mii )a-1 D.. which leads to 6maz = 0 [V] a
Oomns a-1 (a - 1)0a-1 11--100 62-a
min a-1 = [ where |V] is the number of nodes. After
L
Proposition 2. For KNN graph, have EvEv CID(T) = Evev Co(u) - Ever Coo(e) 2 81 - k|V], where |3| S kVVl
Proof. In KNN graph, we have CD(v) CTD(U) + CoD(v), and EvEy CD(v) 2 2/. For Ve E V, CoD(v) = k. Considering that two nodes may regard each other as its nearest neighbors, the total number of edges is at most k|V),, [g] < k.V) Based on the above points, we can derive: Evey Cro(v)= Eesy Cp(u) -vvey Con(v) 2| k|V,, where |3| S k|Vl
12

---

Published as a conference paper at ICLR 2025
Table 5: Performance comparison of different models using different types of GNNs.
Labeling Rates (labels)
2.58%((70)
(.17%
7.75%)(21)
15.51%((20)
22.16% 22.16%((080)
GCN
4..099
0.6038
4o.064
0.6936-a.uus
0.7114+0.000p
SAGE
0,5660
0..994
0.6326+0.0150
0.6910-a.v1or
0.704440.0163
GAT
0.6050 0052
0.6560 0.6262
0025
0.6852_
0092
0.694440.0117
(EGG(GGN)
0.60886 40.0140
0.63I410.0191
0.666120.. 0206
0.69.T2_o.0101
T.7:20.T10001.5.
(PGG(SSAB)
0.5794-1..000
0.6706 0..50220.0100
0.000
0.7056_0.0135
0.723410.0150
SFGL (GAT)
0,597444.0011
0. 00.000
0.699820.0078
0.7148_0.o200
0.7196440.009
SEGL+GPI (GCN)
0,6348
0, 6685640.0131
0.6992+0.com
0.7278_0.0041
0.737440.000
SEGL+GPT (SAGE)
0,6344240.01..
0.6806gg..1:2
0.7050
000000
0.7276_0.n1et
0.7348
SPGL+GP (GAT)
0.6390 17390440..220 0223
0.6654
0.7128 0.0199
0.7256 2.2256-0000144
0.7390.10.0070
DeBERTa+GON
0.6232220oopm 000004
0.661610.0077
0.6728
0.7064_0.0077
0.721440.0090
DeBERTa+SAGE
0.5882+20.00s
0.6396-g0.0100
0.6652 =0. 0090
0.7122_0.007 00.4
0.722440.009
ReBERTa+GAL
0.611244o 0001000000077
16553000.0z00
0.6986#0.012
0.7074-u.v1zt
0.7186
COBERTa+GPT+GGE
0.6588 40.011
0,6978
0.6962 + 0079
0.7206
0.7426
DeBERTa+GPT4SAG
0.6500 40.0113
0.6918 0.6918+o.ousa
0.6914 0.0
0.7166
00163
0.7312 0080
DeBERTa T
0.6482
0.6910
0.711 0.111420.00 0066
0.7118_ 0.059
0.7294 0051
A.4 ADDITIONAL EXPERIMENTAL RESULTS
As mentioned in Sec. 3.2, various GNNs can be used to implement the pseudo-labeler. To assess their impact, we evaluate model performance using different GNNs, including GCN, SAGE, and GAT. Table 5 presents the comparison results across these variants. Our findings indicate that the proposed models exhibit robustness to the choice of GNN, as they achieve similar performance improvements regardless of the specific architecture used.
In Sec. 4.1, pseudo-labeling. In fact, a variety of text encoders can be utilized for this task. To evaluate their impact, experiment with several text encoders, including traditional TF- IDF encoder (Enc1), a pretrained DeBERTa (Enc2), a pretrained 35-Mistral-7B-Instreee (Ene3) (Wang al., 2024), pretrained bge-en-iel (Enc4) (Li et al., 2025), and a fine- tuned DeBERTa (Enc5). Table 6 presents the comparison results on the PubMed dataset. As observed, pretrained models such as De- BERTa, E5-Mistral-7B-B Instrrut,, and bge-en- iel fail to achieve satisfactory results. In contrast, the fine-tuned DeBERTa model demonstrates significant improvement, with TF-IDF performance falling between that of the pretrained and finetuned models. How- ever, finetuning DeBERTa requires suffi- cient number labels, which is why we ini- tially use shallow text encoders to facilitate pseudo-label generation. Notably, the two variants of our proposed SFGL listed in Table 2 use the finetuned DeBERTa their text encoder. As shown, incorporating a more refined text encoder leads to further performance gains. Addi- tionally, both pretrained 25-Misstal-77B-Innttee model but fall short of the finetuned DeBERTa. It is reasonable to infer that a finetuned -5-Mislall- 7B-Instruct or bge-en-icl model would surpass the finetuned DeBERTa in performance.
experiments employ traditional text encoders for graph construction and
Table 6: Results across different text encoders.
Labels
60
120
180
360
500
8ncl
0.665
0.696
0.748
0.749
0.758
Bnc2
0.475
0.518
0.534
0.538
0.548
Bnc3
0.550
0.614
0.648
0.686
0.690
Enc4
0.596
0.636
0.648
0.652
0.667
Enc5
0.743
0.815
0.856
0.877
0.876
and bge-en-icl outperform the pretrained DeBERTa
Figures 6 and 7 illustrate the degree distribution and fitting curves for k = 20. As shown, the degree distributions of KNN graphs with a larger still approximate scale-free structure. We further analyze the KNN graph on the Cora dataset using Euclidean distance as the similarity metric. As depicted in Fig. 8 (b), the degree distribution of the KNN graph (A- = 5) constructed with Euclidean distance does not follow power law pattern. Notably, 2, 080 nodes have zero in-degrees, meaning they are not among the top nearest neighbors of any nodes. This phenomenon likely arises from the way text embeddings encode semantie and syntactic information, which simple Euclidean distance fails to capture effectively. In other words, raw Euclidean distances between embedding vectors do not adequately reflect semantic similarities. This is because Fuclidean distance primarily measures
13

---

Published as a conference paper at ICLR 2025
Algorithm Scale-Free Graph-Language Model (SFGL/ (SFGL/SFGL+GP) SFGL+
Require: Input sequences X, labels L. the number neighbors k. query question q, prompt pt.
1: Encode the text sequences of nodes into hand-crafted features: F = Enc(X).
2: Construct a KNN graph with Cosine distance metrie:
KNN (F, k).
3: Train GNN (GNNg)) on F, A, and y, with loss function in Eq. 1.
4: Generate pseudo labels for the unlabeled nodes: y; = argmax argmax(GNNaa (GNN (f,, A)). E (1, ,n).
5: (Optional) Generate textual responses from an LLM r, LLM(P:). Pi concal (x:, q).
6: Finetune a LM LMg using loss function in Eg. 2.
7: Generate text embeddings: e, LMe(X; (x,) (If LLM is used, replace x, with r,).
8: Train another GNN (GNN,) on E = [1,- em+ n A, and y, with loss function in Eq. 1.
9: Classify unlabeled nodes using GNN y, argmax(GGN (ej, A)), (I....
10: return Yu= 1. .v.-
Table 7: Performance comparison of different models using complete labels.
Dataset
Cora (140)
Cora (1208)
PubMed (60)
PubMed (18217)
GCN
1.60384a.00l
0.7182
0,6770 20770=0.00m 0217
0.8092 10.0057
DeBERTa
0.3048
0.7366
0.3616 2 0.0097
0.9510
DeBERta++PT
0.5430aa.1121
0.7430++0.000
0.4514 000000
+0.0000
SFGL
0.631420.0191
0.727820.0151
0.7238 2o.0122
0.938240.0078
SFGL-GPT
0.685620.0131
0.739610:0105
0.86440=0.0381
0.9400
linear separation between points while disregarding the angular relationships vector orientation, which are key factors in capturing semantic meaning.
Besides KNN graphs, our training framework can incorporate any LGI method to generate pseudo- labels. However, unlike KNN graphs, existing I.GI methods typically require specialized optimiza- tion strategies with complex constraints and additional training the entire dataset, leading to increased computational and model complexity. Morcover, ensuring that graphs generated by LGI methods exhibit scale-free structure is challenging. For instance, SLAPS (Fatemi al., 2021) quires several hours of training to generate graphs, whereas KNN graphs can be constructed in just few minutes. Additionally, as shown in Fig. 8 (a), the graph produced by SLAPS does not exhibit scale-free properties, further highlighting the efficiency and structural advantages of KNN graphs.
Additionally, we evaluate the performance of models using the full set of available labels on the Cora and PubMed datasets. According to the standard data split, there are 1, 068 labeled samples in Cora and 18, 157 labeled samples in PubMed that remain unutilized during training. To assess the impact of complete supervision, we incorporate these additional labels, expanding the training sets 208 and 18, 217 labeled nodes for Cora and PubMed, respectively. Table presents the results comparing the standard and complete label settings. With full supervision, the standalone DeBERTa model achieves strong performance, eliminating the need for pseudo-label generation, as the LMs already produce high-quality text embeddings. However, under the standard split with limited labeled nodes, generating lscudo-labels remains crucial for enhancing performance.
A.5 TRAINING STRATEGIES
The training algorithms of SPGL, SFGL+GPT, SFGL+GCN and SFGL+GCN are outlined in Algo- rithms and 2. The primary difference between SFGL and SFGL lies in steps 5 and 7, where SFGL+G2T T incorporates GPT responses to enhance text embeddings. Similarly, the distinction be- tween SPGL+GCN and SFGL+GON is that the former employs LM to generate text embeddings, while the latter further refines them using LLM. Specifically, SFGL+GCN replaces the original node textual sequences with GPT-generated responses for finetuning. All these models use a GCN as the final classifier.
14

---

Published as a conference paper at ICLR 2025
Algorithm 2 Scale-Free Graph-Language Model (SFGL+GCM BAAGCC//F/GGTTC SFGL+
1: Return to step of Algorithm I, replace F with E generated in step 7.
2: Repeat steps 2, 3, and of Algorithm once generate better pseudo labels.
3: Repeat steps 6, 7, 8, and of Algorithm once to re-classify the unlabeled nodes.
2.5k
Number dogress
Curve in Degree Discribttin
Curve of Degree Distributon
Nimber of nooes with degred"
Curve of in Degree Discribution
80
Kactimal Inellegrex
Meximall InDDegoe
Mazimmll Instlerrs
Number of Modes
300
Number of Noces
2.0k-
Number of Nodes
250
60
159
1.0k
29
Nimber
0.3k
50
Number 150
Number
(a) Cora
(b) Pubmed
(c)
Figure 6: In-degree distribution of KNN graphs on different datasets.
12-1
10
12-1
Proaaiityppo
22-1
Probaaiityy 246)
Probaliltyy s/d)
1072
22
30"
12-7
22-0
12-4
Pibbablitt Density
Procabiltty Dendity
Probability Density
32"1
Exponential
Power-Law
Exconent
PowerLaw
10
Roweer-Law
Expooential
In-Degee
In-Degren
In-Degge
(a) Cora
(b) Pubmed
(c) ogbn-arxit
Figure Curve fitting to the in-degree distribution of KNN graphs on different datasets.
Number of nodes With degroos
Curve of Degree Distri bution
Number nodes degrees
Number Nodes
400
Number of Nodes
10a
300
10a
200
10a
100
10a
20
30
40
50
60
70
0
200
400
600
800
Number of degrees
Number of degrees (d)
(a) SLAPS
(b) Euclidean
Figure 8: In-degree distribution of graph generated by (a) SLAPS and (b) KNN (Euclidean).
A.6 LIMITATION
Due to the inherent differences in the formation mechanisms of KNN graphs and real-world net- works, constructing fully exact scale-free KNN graph remains challenging. A potential solu- tion is to incorporate the scale-free property as constraint within an optimization objective and then optimize it to learn an exact latent graph, which presents an intriguing direction for future re- search. Furthermore, this work primarily focuses citation networks. Extending findings and methodologics to other types of networks, such as social and biological networks, represents another promising avenue for future exploration.
15


Reviews:
[{"review_id": "RbhL9ATbFn", "reviewer": "Reviewer_NtXH", "summary": "This paper proposes a graph-based method as a semantic complementary supervision for text-attributed graphs. The author first theoretically justifies that a kNN graph can approximately describe a scale-free network while fulfilling three constraints or properties. Based on this, the author proposes a kNN graph construction to initialize the document graph. Then, the author introduces a new iterative method that fuses embeddings from three sources: a pseudo-labeler, GCN embeddings, and text embeddings.", "questions": "Introduction: \n0. Clarification of Focused Downstream Task: The specific downstream task of your proposed model is not clearly articulated in the introduction. \"When developing a graph language model (GLM) for classification tasks\", it is important to clearly define the target task. Typically, downstream tasks are categorized at the node level, edge level, subgraph level, or graph level. In machine learning, these tasks include classification, regression, clustering, and other related objectives. Please specify the precise downstream task your work focuses on. If it is interdisciplinary, make sure to distinguish it from existing topics and clarify its unique contributions in comparison to related fields. If the graph is directed graph please be clear about it too. adapt the section graph construction too. Directed graph should have +1 -1 but not binary for instance ? \n\n\n0. \"However, LGI models typically rely on artificial assumptions about the underlying edge distribution, which may\nnot accurately reflect the real graph structure.\" Regarding a series of gcn-based methods, please be concrete about the underlying assumption and why it is too strong in practise or theory?  artificial assumption should also be shortly mentioned with a reference. \n\n0. However, finetuning a pretrained LM typically requires target datasets with extensive annotations (Brown et al., 2020), which is an unrealistic requirement for semi-supervised learning tasks and may make LM finetuning unreliable if this requirement is not satisfied.  Please list out several publications or provide an example of computation demand? Please refer to https://arxiv.org/abs/2401.15569, https://arxiv.org/abs/2305.19523. \n\n0.  We identify two key challenges in existing GLMs: artificial structural assumptions for graph generation and unreliable LM finetuning for text embedding. https://academic.oup.com/book/27303 Please refer to this book, especially the estimation of alpha. Please provide an alpha estimation based on their method to identify the condition of validity and effectiveness. Theoretically any graph inhabits some extent of the hierarchy. But only alpha \\in [2, 3) would be considered as scale-free is it? Please be clear about the limitation of this assumption https://www.nature.com/articles/s41467-019-08746-5. \n\n\n\"To the best of our knowledge, this work is the first attempt to integrate graph generation and text embedding into a unified GLM framework. \" \nIt would be better to shortly review previous assumption and their limitation. It is also necessary to prove whether such a prior is more complete than the previous one？Please also add some sentences about the contribution involved and why graph generation is crucial, from which perspective? performance improvement, identify the hypothesis gap? \n\nFigure 1, if the discussion is about scale-free network, it would be better to plot in log-log sorted degree template? \n\nAlpha could quite efficiently estimated https://academic.oup.com/book/27303, please also consider it to estimate k? \n\nin Section 2.3  The degree distribution of KNN graphs is primarily shaped by three critical factors: ① node distribution, is it edge distribution or node? \n\nSection 3.1 There is still some space to quantify such estimation of k, if the adjacency matrix is know, it should be possible to provide a approximation error? \n\n\n2. Regarding the graph's underlying assumptions, the homophily assumption is another important one. Would it not be necessary to compare your approach with this assumption in the related work? Please explain how your method relates to or differs from approaches based on homophily from various level, application, dataset setting?\n\n3. If this work has outperformed the state-of-the-art (SOTA), please highlight it in the introduction and abstract with margin. If not, it would be interesting to mention the gap and your specific focus.\n\n4. If GCN, LM and kNN graph is involved it would be necessary to provide an analytical time complexity analysis and if possible empirical space complexity analysis, so that reader could know whether it is a method with better complexity/performance ratio or refresh the upper bound of current algorithm without taking complexity and scalability into consideration.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The author has focused on a topic with wide applications, going beyond GCN for text-attributed graphs. The proposed method can be applied to any document without explicitly extracting links.\n2. This paper theoretically proves that k-nn can approximate the degree distribution of a scale-free network.\n\n3. The paper presents an extensive set of experiments with several reference citation networks (Cora, Pubmed, ogbn-arxiv and arxiv23). Comprehensive experiments have been conducted using different labeling fractions under semi-supervision conditions and a different number of neighbors in the KNN model, with a detailed description of all hyperparameters used.", "weaknesses": "1. Contribution and Achievements are not clearly introduced: To the best of our knowledge, this work is the first attempt to integrate graph generation and text embedding into a unified GLM framework.\n\n2. Algorithm Design: The introduction of the algorithm design should include necessary details in the main paper, not just in the Appendix. The roles of the GCN, text embedding, and pseudo-labeler should be explained in the main paper. A complexity analysis of each step should be provided shortly in the main paper too. \n\n3. Experimental Results: Clarify whether the results from TAPE and GLEM are directly taken from their respective papers. Additionally, there are other important baselines such as GraphFormers, Leading, SimTEG, and Eingne. If possible, please discuss or compare your results with these models as well. You may also refer to related work in this paper: [https://www.ijcai.org/proceedings/2024/0634.pdf](https://www.ijcai.org/proceedings/2024/0634.pdf).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HxX8KRke5O", "reviewer": "Reviewer_dcB6", "summary": "The paper focus on node classification of text attributed scale-free graph. The authors discovers that citation networks have the property of scale-free graphs, and proposed KNN Graph based on text similarity distance.  The authors theoretically and empirically demonstrate that KNN graph has property of scale-free graph and can be used as a good prior for scale-free graph such as citation network. \n\nIn the experiment, the authors showed that with small size of labeled data, the model can outperform both GNN based models and Pretrained Language Model based baselines. Ablation studies are performed to demonstrate the effectiveness of KNN Graph approximation for scale-free graphs.", "questions": "1. Correspond to weakness 1, in the paper between line 103 - 209, the paper discuss why KNN graph is similar to scale-free graph. The property of Homophily and Directed seems trivial.  Upper bound of degree is proved using Proposition 1 and Proposition 2, but it can not explain why KNN graph's node degree follow the power law which is an important property of scale-free network. Empirical curve fitting shows that probability of node degree of KNN graph could follow the power law, is there any theoretical  explanation behind this ?\n\n2. I find that there are only experiments based on academic citation network. But in the appendix, some analysis on social network data are done, is there any experiment for the task on social network ? It can be more convincing if experiments on other domain can be performed", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposed a new KNN Graph, which has the property of scale-free graph, can be used as a good prior of scale-free graph such as citation graph. \n\n2. The paper first introduce the application of scale-free graph in text attributed citation network.\n\n3. The proposed approach is carefully verified using empirical and theoretical approach. \n\n4. The proposed method outperformed strong baselines \n\n5. The experiments conducted by the paper is solid and comprehensive\n\n6. Detailed ablation studies are done to analyze the impact different components. Including different text encoders, different graph neural network \n\n7. The paper is clear and easy to follow", "weaknesses": "1. The explanation of correlation between theory of scale-free network and KNN graph is a bit insubstantial, the theoretical explanation did not provide sufficient intuition to the approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I69WdU7iZY", "reviewer": "Reviewer_mKrX", "summary": "The paper introduces a new framework that combines graph generation and text embedding using the scale-free property of real-world networks. By approximating this structure with a KNN graph, the model improves graph generation and enhances language model finetuning. The paper uses a graph-based pseudo-labeling method to improve performance on semi-supervised tasks, validated through experiments on several citation network datasets​", "questions": "1. I believe validating KNN is crucial, and I suggest the authors include comparisons with other graph generation methods, such as random graph generation or rule-based approaches. \n\n2. I am curious how the specific implementation of real graph in Table 6 is set up?\n\n3. I noticed that the real graph is not even as good as the graph generated by KNN, I hope the author can give your thoughts on this. Such results give people the inspiration that the graph generation is not for fitting the real network connections, but for the information transfer of nodes with high similarity. However, this ignores the information transmission of real edges, because nodes only believe the points that are close to each other, which is easy to introduce errors for complex network mechanisms.\n\n4. I think a deeper experimental exploration of the above questions should involve trying LLM-only prediction ways in the network, such as [1] [2], etc., missing in the paper cited.\n\n5. The paper lacks some experimental details, such as the division of datasets into labeled and unlabeled nodes (m, n) and the specific settings of the iterative algorithm. \n\n[1] Bi B, Liu S, Wang Y, et al. Lpnl: Scalable link prediction with large language models[C]//Findings of the Association for Computational Linguistics ACL 2024. 2024: 3615-3625.\n\n[2] Ye R, Zhang C, Wang R, et al. Natural language is all a graph needs[J]. arXiv preprint arXiv:2308.07134, 2023, 4(5): 7.\n\nConcise and powerful responses will help me understand better, and I will consider raising my score.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea is novel. A unique contribution is to use KNNs for Graph generation, which I think will be very inspiring for semi-supervised tasks in homogeneous networks, especially from the comparison with the Real Graph on Table 6, the generated graph is even better than the real graph.\n\n2. The paper is well-written, providing a detailed description of the problem setting and research background.  The authors clearly articulate the challenges in existing graph-language models and their motivation for the proposed approach, making the rationale behind their work easy to follow.\n\n3. The extensive ablation studies included in the paper significantly contribute to the credibility of the proposed method.", "weaknesses": "1. The paper primarily focuses on citation networks, which are inherently homophilic.  The applicability of the proposed KNN-based graph generation method to other types of textual networks, such as academic or social networks, is not explored. \n\n2. The core contribution of the paper lies in using KNN to approximate graphs, but this approach seems to be more suited for homophilic networks.  In more diverse scenarios, such as heterogeneous academic networks with different types of nodes, the use of a unified nearest-neighbor metric may introduce significant bias.  This limits the method’s applicability to broader graph structures. It is recommended to try a broader extension of the for KNN design.\n\n3. While the paper provides strong theoretical and experimental support, the experiments on the validation of KNN could be more comprehensive. (See Questions)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C3W5sXhgcP", "reviewer": "Reviewer_7hx6", "summary": "This paper proposes a novel scale-free graph-language (SFGL) model. There are two main components in the proposed framework: (1) Scale-free KNN graph construction; (2) LM Training augmented with KNN graph-based pseudo labels. Experiments demonstrate that SFGL shows comparable performance.", "questions": "See Weeknesses part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The use of scale-free properties for graph generation is clever and practical. The approximation via KNN graphs is a simple yet effective method.\n2. While the use of pseudo-labels is not a new concept in integrating graphs and language models, the way it is applied in this paper is particularly effective.\n3. The empirical results are compelling, with the SFGL model consistently outperforming baseline methods across various datasets and labeling rates.", "weaknesses": "1. The author could conduct more experiments with different values of labeling rates. Although the paper focuses on the low labeling rates, it will be meaningful to extend the experiments to higher labeling rates (e.g., 50% to 60%) to better understand the model’s scalability and performance under such higher labeling conditions. Observing the model's behavior with more labeled data could also provide insights into whether the proposed improvements hold up when labeled data is no longer a significant limitation.\n\n2. Although the paper demonstrates that KNN graphs have comparable performance to the real graph (Table 6),  the differences between KNN and real graph structures are not thoroughly explored. A more detailed case study or analysis comparing the graph structures between KNN graphs and real graphs could provide valuable insights.\n\n3. Some minor points:\nIn Algorithm 1, Textual feature X -> Text sequences X, consistent with line 265\nIn Algorithm 2, replace X with E -> replace F with E. (X are input text/tokens, E and F are text embeddings.)\n\n4. In abaltion studies, the author could use some more powerful text encoders, especially some recently used LLM-based encoders, such as E5-Mistral-7B-Instruct (https://huggingface.co/intfloat/e5-mistral-7b-instruct). More powerful encoders can be found in the MTEB leaderboard (https://huggingface.co/spaces/mteb/leaderboard). \n5. In ablation study section, the author could report the results when only using KNN graph generation without the LM-finetuning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)