<SO$G_k$>: One LLM Token for Explicit Graph Structural Understanding
Authors: Jingyao Wu
Bin Lu
Zijun Di
Xiaoying Gan
Meng Jin
Luoyi Fu
Xinbing Wang
Chenghu Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: LLM for Graph
Graph Structure Learning
Structure Hallucination
Primary area: learning on graphs and other geometries & topologies

Abstract: Large language models show great potential in unstructured data understanding, but still face significant challenges with graphs due to their structural hallucination. Existing approaches mainly either verbalize graphs into natural language, which leads to excessive token consumption and scattered attention, or transform graphs into trainable continuous embeddings (i.e., soft prompt), but exhibit severe misalignment with original text tokens. To solve this problem, we propose to incorporate one special token <SO$G_k$> 
to fully represent the \textbf{\underline{S}}tructure \textbf{\underline{O}}f \textbf{\underline{G}}raph
within a unified token space, facilitating explicit topology input and structural information sharing. Specifically, we propose a topology-aware structural tokenizer that maps each graph topology into a highly selective single token. Afterwards, we construct a set of hybrid structure Question-Answering corpora to align new structural tokens with existing text tokens. With this approach, <SO$G_k$> empowers LLMs to understand, generate, and reason in a concise and accurate manner. Extensive experiments on five graph-level benchmarks demonstrate the superiority of our method, achieving a performance improvement of 9.9–41.4\% compared to the baselines while exhibiting interpretability and consistency. Furthermore, our method provides a flexible extension to node-level tasks, enabling both global and local structural understanding. The codebase is publicly available\footnote{The code of our project is available at \href{https://anonymous.4open.science/r/SOG-8432}{https://anonymous.4open.science/r/SOG-8432}.}.

Published as a conference paper at ICLR 2026
<SOgk>: ONE LLM TOKEN FOR EXPLICIT GRAPH STRUCTURAL UNDERSTANDING
Jingyao Wu', Bin Lu Zijun Dit, Xiaoying Gan', Meng Jin', Luoyi Fu', Xinbing Wang', Chenghu Zhou²
Shanghai Jiao Tong University, PIGSNRR, Chinese Academy of Sciences
(wujingyaad wobinnul2000. 200, j75du, ganxiaoying, jinm, yi yiluuf, xwang8] es jsjtu.ed cn, wwwuchelreis.comm en
ABSTRACT
Large language models show great potential in unstructured data understanding. but still face significant challenges with graphs due to their structural hallucina- tion. Existing approaches mainly either verbalize graphs into natural language, which leads to excessive token consumption and scattered attention, or transform graphs into trainable continuous embeddings (i.e., soft prompt), but exhibit se- vere misalignment with original text tokens. To solve this problem, we propose to incorporate one special token <SOG,>> fully represent the Structure Of Graph within a unified token space, facilitating explicit topology input and structural in- formation sharing. Specifically, we propose topology-awars structural tokenizer that maps each graph topology into a highly selective single token. Afterwards, we construct a set of hybrid structure Question-Answeriin corpora to align new structural tokens with existing text tokens. With this approach, <SOG,> empow- ers LLMS to understand, generate, and reason in a concise and accurate manner. Extensive experiments on five graph-level benchmarks demonstrate the superior- ity of our method, achieving a performance improvement of 9.9-41.444 41.4% compared the baselines while exhibiting anterretabbiitty and consistency. Furthermore, our method provides a flexible extension to node-level tasks, enabling both global and local structural understanding. The codebase is publicly available"]
INTRODUCTION
In recent years, Large Language Models (LLMs) have demonstrated impressive capabilities in the general understanding of unstructured text (Guo et. 2025). However, there are still significant lim- itations in how L.MMs comprehend text-attributed graphs, such as knowledge graphs (Wang et al. Lin et al. 2025), citation networks (Chen al. 2024 Tang al. 2024). transaction net- Lei et al., 2025). Compared to LLM--a--nhancerr approaches (He et al. 2024a Liu et al,, L.MMas-Predicttor methods (Ye et al. 2024 Tang et al. 2024] Chen al. Huang et al., (024) gain more widespread attention due to their generalizability across dif- ferent level tasks and datasets/domains. Thus, for the serialized input format of LLMs, the core challenge in understanding graphs lies in how to feed the structure into the model in a complete, accurate, and efficient manner.
2025b
works
2024
et al. 2024b).
2024
Existing works mainly transform the graph topology in two ways: one is Graph-to-Text, and the other is Sraph-to-Embedddee
(I) Graph-to-Text methods (Ye et al. 2024 Zhao et al., 2023b Fatemi al. 2024) directly verbalize graph topology with natural language and are widely used in prior work. Yet, due to the increasing scale of wool-wordd graphs, depicting the complete topology in text consumes large amounts of tokens, which scatters LLM attention and impairs holistic struc- tural understanding. In addition, existing work (Cao et al. 2025) reveals that LLMS exhibit strong structural hallucination, that even small permutations in node/edge orders can yield different output, inducing unstable structural understanding.
(2) To reduce token overhead, Graph-to- -ote---ddmm
methods (Tang al. 2024 Chen et al. 2024 Huang al. 2024 Liu al. 2025a) compress graph
"Correspondence to: Bin Lu (robinlul I 209 sjt@sjt..cdu.c)
'The code of our project available http://gtthbbbcomminggaaWWWVVVV

---

Published as a conference paper at ICLR 2026
topology into continuous embeddings and project them into the LLM. However, it inevitably intro- duces a modality mismatch. The projected graph embeddings are poorly aligned with LLM inherent token space, leading to weak cross-modal compatibility and degraded structural comprehension. Therefore, it raises the following question: How to concisely and accurately feed graph topology information into an L.M avoid structural hallucination?
To solve the above problem, we propose to incorporate only one special token for LLM, denoted as <SOgE>, fully represent the Structure Of Graph within the same space with text tokens. <SOG>>> facilitates explicit topology input and structural information sharing. To be specific, we firstly pro- pose a Topology-Awan
Graph Structural Tokenizer encode each graph topology faithfully and pull one structural token from discrete token vocabulary. This compact yet expressive token encap- sulates critical topological information while demonstrating a highly selective property. To align <SOGE> with L.MSs textual input, we then construct set of Hybrid Structure Question-Answerti (QA) that combine text tokens and structural tokens within a task agnossic context. After tuning on these novel corpora, it guides the structural tokens to live in the same space with text tokens, hereby promoting the LLM to seamlessly read and understand topology.
To summarize, the main contributions of our work are as follows:
We propose to utilize only one structural token <S09E> obtained by a topology-aware graph tokenizer to effectively provide structure information to the LLM in a highly selective manner.
To align the embedding space of two types of tokens, we design a set of hybrid structure QAs to instruct LLM to understand, generate and reason the structure, which harmonizes text and topology for unified understanding.
Extensive experiments on five graph-level benchmarks demonstrate that our method outperforms all the baselines with performance improvement ranging from 9.9% to 41.4%, while exhibiting strong interpretability and consistency. Additionally, our method can be effectively extended to node-level classification, showing <SOG,>> > possesses both global and local structure perception.
2 RELATED WORK
Recently, LLMs show promising performance in understanding structured graph data, where the core issue is how to input non-Euclidean graph into serialized LLMs. Existing work can primarily be classified into two categories: Graph-to- toxt methods and to-beddding methods.
Graph-to-Text methods verbalize the graph using natural language, transforming the structure into a text sequence, which are widely adopted in existing works. The most straightforward methods (Ye) al. 2024, Wang et al. 2023) describe graph connectivity by enumerating edges and indirect in the form of "node A is connected to node B" along with edge attributes. Besides, Li et al. (2024a) and Wang et al.] (2023) directly feed adjacency lists in text form into the LLM. To leverage the commonsense knowledge of LLMs, Talk Like a Graph (Fatemi et al. 2024) characterizes nodes and edges as various types of real-world connections, such as friendships, co-authorships, and character relationships. InstructGraph (Wang al. 2024b) formulates a graph Python class with nodes and edges as attributes, leveraging LLMs code reasoning abilities. To further improve traverse efficiency, Graph'Text (Zhao al., 2023b) describes k-hop neighbourhood structure of central node with neighbouring features and labels, enlarging the receptive field of L.Mss. LangTopp (Guan) al., (024) innovatively constrains the LLM's output and aligns the LLM comprehension of text- formatted graph inputs with the GNN's structural understanding at quantization level. Recently, Dr.E (Liu et al. 2025b) summarize each hop information into multiple textual tokens in existing LLM vocabulary. However, all these Graph-to- teapttt-TTett methods require large token consumptions to fully depict the graph structure, scattering the model attention for an overall structural understanding. Moreover, LLMs are highly sensitive to variations in the order of node/edge description, leading to unstable structure reasoning (Cao al. 2025).
Graph-to-Embedddin methods compress the graph into embeddings and then project them into the token space of the LLM, which also named 'soft prompt' in some works. GraphGPT (Tang et al. 2024) encodes textual and structural features separately, learns hybrid node embeddings via contrastive learning, and maps them into the LLM token space through an MLP-based projector. To reduce structural encoding complexity, LLaGA (Chen et al. 2024) adopts Laplacian embed- dings and parameter-free node aggregation. Moving beyond node- or hop-level representations, G-retriever (He et al. 2024h) encodes retrieved subgraphs and pools them into graph-level embed-
2

---

Published as a conference paper at ICLR 2026
Graph Vocabulary
Hybrid Structure QAs (Sec 4.2)
neighoor
Node
K-NN match
TF Jutgement
Dearr piann Matchh
GNN
Large Language Model
Graph Topology
Topology Aware Graph Structure Tokenizer (Sec 4.1)
Task Pronpe
Textuul
<B0G
Figure The overall architecture for LLM understanding with structural token <SOG>>.
dings, which are then adapted to the LLM input space via an MLP. Several works further enhance structural expressiveness: T'A-GLMM (Wang et 20244) constructs contrastive views through edge deletion and node masking, while GraphLLM (Chai et al. (023) employs a shared MLP to project graph embeddings into the input space of each LLM layer. Instead of using MLPs for projection, GraphAdapter (Huang et al. 2024) leverages a GNN- based projector to improve efficiency on text- attributed graphs. Beyond direct projection, TEA-GLM (Wang et al. (024a) and GNP (Tin et al., 2024) additionally apply PCA-based mapping or attention- based pre alignnigmeent before space align- ment. In spite of these success in compressing structure information with less token consumptions than Graph-to- Text methods, between LLM inherent token space and projected graph embeddings.
methods still suffer from severe misalignment
To sum up, our approach aims to utilize only one structural token, i.e., <SOG,>, sharing the same space with LLM inherent tokens, to fully characterize the graph structure. In this way, we follow a new Graph-to- Token paradigm, enabling harmonious token mapping and understanding across graph structure and textual attributes within LLMs.
PRELIMINARY
Given acxt-attributed graph, it contains graph structure G (V, E), corresponding text information T, and label y. where V {u², v² ...., vlVij } the set of nodes, and E' C V x V is the of edges. indicating the presence of edge (u², w)). We take L.M M (1;),) O) for graph understanding, where V denotes the vocabulary, and θ denotes the parameters. In our work, we propose a group of structural tokens {<S0g)>, <SOGK>> explicitly reflect graph overall structure information, where K is the number of structural tokens. Each structural token <SOG>> represents a prototype of pure graph structures. These structural token share a same token space with LLM original vocabulary V, hereby forming a new vocabulary V = V U TUUf&SoGs (<SOG1>,". <SOgx>}.
The adjacency matrix of G A (0, 1IIV//V,, with A,
Afterwards, to obtain the answer ý using LLM as predictor, the input to M (:: VM, +) consists of: (1) task prompt P with task requirements, desired output format and candidate labels, (2) textual graph attributes T, (3) corresponding structural token <SOGg>, Thus, our objective is let LLM M generates the output O containing the predicted label ý C O that matches the ground truth label y. Formally, this process can be denoted as follows:
O M([P,T,
+SOG*>HH;>> (M(P, θ).
This formulation allows structural information to be seamlessly integrated into the generative rea- soning process of LLMs, thereby achieving more reliable inference graph data.
METHODOLOGY
In this section, we formally introduce how to generate and utilize the structural token, i.c. <SOG,>, to enhance ILM'' graph understanding in a two-stage manner as shown in Figure I Specifically, in the first stage, we propose topology-awer graph structural tokenizer, which extracts the graph topology, encodes global information, and further projects into one structural token. In the second stage, multiple hybrid structure Question-Answer (QA) pairs are constructed and fed into the LLM to ensure the integration of text tokens and structural tokens in unified space. Thereby, LLM can understand, generate and reason these tokens obtain satisfied results across different graph tasks.
4.1 TOPOLOGY-AWARE GRAPH STRUCTURAL TOKENIZER
Given text-attributed graph, the topology-aware graph structural tokenizer first extracts its graph topology G(V retaining only the pure structural information. To enhance the topology un- derstanding, assign each node ed E V with new structural attribute & with hierarchical
3

---

Published as a conference paper at ICLR 2026
traversal strategy. An anchor node is first identified based node importance measure (e.g., node degree). Then, each node in the graph is located by its relative position to the anchor node, forming its node attribute such as first-hop neighbor #1', 'second-hop hop neighbor #3'. The numbering of nodes within the same hop (e.g., #1, #3) is determined by ranking them accord- ing to the same node importance measure. In addition, connect all nodes, serving as a pooling mechanism to obtain an overall representation. Finally, we adopt an off-the-shelf text encoder fr(+) to embed the newly added attributes: x fr(()), X. = [=];; .....: x globall R([[++)xx, where d, is the dimension of the text encoder. This hierarchical strategy provides a unique spatial coordinate and informative feature for each node, eliminating the vague description induced by node-invariane permutation.
introduce a virtual global node to
Then, to aggregate node information across its multi-hop neighboring, we employ a graph neural network (GNN) fa(.) as the encoding module (Kipf & Welling, 2017). Through message passing mechanism, GNN captures topology-aware information, mapping the node structural attributes X, the latent representation H, in continuous space, where H, = fa(X,), H, [[(++1)xd and is the dimension of GNN encoder. The representation of the virtual global node 1 E R is then taken graph-level structural representation. However, the continuous representation fails to align with discrete text tokens for LLMs, leading to unexpected structural hallucinations.
Therefore, propose to map the continuous representations into K discrete token vocabulary uti- lizing self-supervised topology reconstruction (Yang et al.] 2024a), where K hyperparameter that controls the size of the vocabulary. Specifically, we introduce learnable vocabulary de- noted as C {ei, C2, .,,,,.................... cK which serves as set of discrete latent embeddings for structural patterns. Each entry ej corresponds structural prototype learned during reconstruction task. For each node feature hi compare and retrieve the nearest codebook entry c 2. ne C) based on Euclidean distance:
j = arg min | - c2l22 Cg c= C2, ..,....... cx].
To achieve this, two loss functions, i.e., update loss lupdite = ||sg[hi] Ca l| and commitment loss Coommit = || sg[ce]]l. are utilized to find bridge between eı and hig. In particular, the vocab- ulary index corresponding to the global node is chosen as the structure vocabulary for whole graph. To reflect its lopoloy--awarenss. the selected vocabulary entries enable the topology recon- struction with a lightweight decoder. Thus, we incorporate a shared decoder module f-)-) to obtain reconstructed node features X = (z,)), where ze (H,) C CIVI+I denotes the selected vocab- ulary entries corresponding the node features H3. X is then used to reconstruct the adjacency matrix À X thereby computing the reconstruction loss between Á and original A. Thus, the overall loss function consists of three components, and is further denoted as follows:
A-A A
[|sg[H.] ze (H&)) +B 1 sg|z (H.
Update loss
Commitment loss
Reconsiiaction loss
where / is the weighting hyperparameter.
4.2 TOKEN ALIGNMENT WITH HYBRID STRUCTURE QAS
After obtaining the graph structural token, requires to align with original LLM text tokens for a unified understanding. To solve this problem, construct hybrid structure QAs to assign reason- able embeddings for newly-added structural tokens and tune the LLM encoder for context awareness. We design types of corpora for the structure QAs, which are illustrated as follows:
1. k-Nearest Token Neighbour Matching:
The question is "Here is the target structural token <SOG,>, and its five nearest graph structural lokens are:". Afterwards, the ideal answer its Top-k nearest tokens based on vector cosine similarity within the vocabulary. This QA corpus ensures a local agreement with that the closer the discrete structural representation is, the closer the LLM token embedding is.
2. True/False Structure Similarity Judgment: The question is "Here are two tokens <SOg,> and <SOG>>, judge whether they represent similar structures not." If the cosine similarity
4

---

Published as a conference paper at ICLR 2026
between two corresponding global node embeddings in the continuous space is greater than predefined positive threshold, the answer "similar"; if is smaller than a negative threshold, the answer is "dissimilar". This QA corpus leverages the idea of contrastive learning to clarify the boundary of structural similarity in token embedding space.
3. Description-Toke Pairs Matching: The question is "Here is the target graph: node and node B is connected; node A and node C connected ..... The corresponding graph structural token is:". The answer is its true structural token. This QA corpus directly link the text token understanding with the structural token understanding, encouraging LLM to explicitly unify two tokens in the same "brain".
Through these QAs, conduct supervised fine-tuning the LLM. (1) To maintain pretrained knowledge, we freeze the pre-existing text tokens, and only update the token embeddings for newly- added structural tokens. (2) To further improve the applogy-awareness we employ a lightweight parameter adaptation technique, i.c., LoRA (Hu et al., 2022), to efficiently inject structural informa- tion into the model. Notably, in this fine- tine-tuning process, the constructed QAs solely depend on the graph structure, neglecting the original textual information or downstream tasks. Therefore, it offers strong generalizability and scalability across different graph tasks and domain information.
4.3 DOWNSTREAM APPLICATION WITH <SOG
To guide LLM for different downstream applications, we conduct a task-specific fine-tuning stage, hereby transforming the task into text generation paradigm. Specifically, we first input system prompt as a general guidance: "Classify target graph into the correct category based on its graph topology (i.e. structural token) and textual attributes. Each structural token represents unique graph pattern (e.g., a prototype of similar molecular structures)." Following S, we append a user prompt to concretize the input information and application scenario. The user prompt of LLM consists of the following three parts: (1) Task Prompt P: Explicitly guide the LLM to understand task objective, such as "[Task] Predict whether the molecule is toxic." or "[Task] Classify the central paper node by assigning it the correct label from the set of candidates... (2) Textual Graph Attributes T Provide original semantic features and background information about the graph, such as '[Molecule] The molecule's SMILES expression is...' or '[Paper] The center node represents paper titled...' (3) Assigned Structural Token <SOG>>: Represent the graph topology using "[Structural Token] <SOG During task-specific fine-tuning, the ground-truth label y for the particular downstream application is incorporated after the LLM input, i.e.system prompt and user prompt, as supervised signal to guide response generation. The loss function is denoted as follows:
L = - Σ log (v yet, P,T, <SOG>>).
where indexes the position in the output sequence. In this way, downstream applications can further align text/structure understanding with different domain tasks.
EXPERIMENT
In this section, we empirically evaluate the effectiveness and structural contributions <S0G,> via extensive experiments by addressing the following research questions (RQs):
RQI: How does <SOGg> perform compared to other baselines graph elassification tasks?
RQ2: How effective the proposed structural tokens <SOGt>?
RQ3: How interpretable are the structural tokens? Do they maintain alignment with text tokens?
RQ4: Can <SOG extend improve L.M structural understanding node-level tasks?
RQ5: How sensitive <SOG>> to key structural tokenizer design choices?
5.1 EXPERIMENT SETUP
To evaluate performance, we perform graph-level classification tasks on 5 datasets from MoleculeNet (Wu et al. 2017). including BBBP, Tox21, ClinTox, HIV, and BACE, covering tox- icology, clinical research, and pharmaceutical chemistry domains and encompassing 17 subtasks. The detailed information is summarized in Appendix A.1
Datasets.
5

---

Published as a conference paper at ICLR 2026
Table 1:
Performance comparison across five datasets, where bold indicates the best performance
and underlined indicates the second-best.
LLM Backbone Method
BBBP (t)
Tox21 (f)
ClinTox (f)
HIV (*)
BACE (t)
GPT-4
Zero-shot
61.5
55.2
51.6
65.9
62.5
GPT-4o
Zero-shot
36.0
51.8
38.5
Deepseek-RI
Zero-shot
63.6
60.1
48.2
50.5
52.7
LLaMA3-70B
Zero-shot
60.1
44.9
48.8
58.3
50,9
Galactica- 120B
Zero-shot
66.1
68.9
82.6
74.5
61.7
58.8
66.0
71.3
62.2
Non-LLM
GPF-GCL
56.5
63.6
49.3
52.1
Molecule
GIMLET
59.4
61.2
66.2
69.6
Approach
MolCA-S
62.5
66.6
69.0
62.8
MoICA-GS
63.6
68.5
72.7
63.9
Zero-shot
50.2
48.5 2.7
5.5
51.8 0.8
52.7
Few-shot (Random)
3.8
49.9 10.2
6.2
49.7 16.7
68.2
Few-shot (Morgan Sim)
2.5
8.7
53.3 10.0
42.0
0.6
LLMMA3-BB
LoRA SFT
6.22
5.8
63.8
0.7
5.95
Soft Prompt
27.9
1.9
3..0
33,6
2.9
<S00,> (Ours)
3.1
8..4
8
75.7 1.6
63.3
Zero-shot
50.0
3.3
2.6
5,00
49.9 0.4
Few-shot (Random)
49.4
49.3
50.3 1.7
4.
47.8 1.8
Few-shot (Morgan Sim)
0.8
2..1
6.38
0.74
53.2
LLaMA2-7B
LoRA SFT
55.0
6..6
5.73
78.1
61.7 2.7
Soft Prompt
53.7
49.8
4..7
44.9
49.4
<SOG,> (Ours)
66.4 2.7
3.14
9.1
83.2
98.4 0.8
Evaluation Metrics. We adopt generative question answering method to obtain the result from LLM, and evaluate by text matching determine whether the correct answer appears in the sponse. The maximum output token length is limited to 100. The LLM sampling parameters are set with lemperature 0.6 which strikes a balance between randomness and coherence, while limiting the token selection to those with cumulative probability mass of 90%. To quantitatively evaluate the classification results, we use AUC-ROC to assess the performance of binary graph classification tasks, most tasks exhibit significant class imbalance. Each experiment is run three times. and we all report the average performance. The detailed information is summarized Appendix A.2
Baselines. We compare our proposed method against 20 baselines, which generally divided into three categories: (1) LLMs with extensive parameters (> 70B), including GPT-4 (OpenAl
2023).
GPT-4o (Hurst et 2024). Deepseek-R1 (Guo al. 2025), LLaMA3-70B (Dubey et al. 2024
Galactica- 120B (Taylos lor et al.
2022); (2) non-LLM approaches. including Atr--atraskiing (Fang
al. 2023 Hu et al.) 2020b).
GPF-GCL (Fang et al.
2023 You al. 2020), GIMLET (Zhao al.
2023a), MolCA-S (Liu et al.
2023b), MolCA- (Liu et al. (023b); ; (3) Methods on LLM with few
parameters (i.e. LLaMA2-7B (Touvron al. 2023). LLaMA3-3B (Dubey et al. 2024)). including
zero-shot, few-shot (Brown et al. 2020), LoRA A-bsed supervised fine-tuning (LoRA SFT) (Hu at.
2022). and soft- soft-prompt (Perozzi et al.] 2024). Detailed information refer Appendix A.3
We illustrate the implementation details of our method in Appendix B. Meanwhile, the prompt designs individual downstream tasks are presented in Appendix B. and additional training configurations and details are documented in Appendix B.2
Implementation Details.
5.2 OVERALL PERFORMANCE (RQ1)
The experiment results on five graph-level datasets are shown in Table We have the following observations: (1) <50G consistently outperforms all the baselines, demonstrating satisfied per- formance improvement from 9.9% to 41.4%. In the BBBP and Tox21 datasets, our method rank first and second in performance on the 3B and 7B models, respectively. On the remaining three datasets, our method on the 7B model achieves the best performance. Notably, <SCG,> attains an AUC-ROC of 98.4, with a significant performance gain over sub-optimal model. Due to the varying levels of comprehension difficulty and the number of training samples across different datasets, the increase in model parameter scale does not result in direct performance im- provements individual datasets. (2) Compared to non-L_M methods (GPF, GIMLET, MolCA),
BACE dataset,

---

Published as a conference paper at ICLR 2026
Table 2: Ablation of different structural token <SOG,>> on five datasets.
Model
Ablation Study
BBBP
Tox21
Clin'Tox
HIV
BACE
(Mla) w/o Structural Token
61.2
72.3
81.8
53.0
53.2
(MIb) Static Structural Token
60.7
67.8
52.5
54.6
54.3
LLaMA3-3B
(MIc) Random Structural Token
57.3
62.2
52.8
55.3
55.8
Ours
76.9
83.4
85.5
75.7
63.3
(Mla) w/o Structural Token
61.3
66.4
79.5
69.6
94.2
(MIh) Static Structural Token
64.7
69.1
92.0
81.5
94.2
LLaMA2-7B
(MIc) Random Structural Token
62.1
68.6
88.5
83.4
96.7
Ours
66.4
72.4
94.3
83.2
98.4
large parameter LLMs, such as Galactica- 120B, exhibit strong text comprehension and achieve bet- ter results on some datasets even in a zero-shot setting. However, due limitations in structural understanding, their performance improvements are constrained. Our proposed structural token ap- proach requires only a single token to surpass the performance of large models on smaller parameter LLMs (3B and 7B), highlighting our effectiveness. Meanwhile, our approach is inherently more cost-effective and resource-efficient. (3) For the baseline methods of the 3B/7B large language models, few-shot approach exhibit a large variance. For instance, the 3B model shows standard deviations as large as 16.7 on HIV and 17.5 on BACE, making the results unstable and unreliable. LoRA SFT method improves performance by explicitly fine-tuning parameters to adapt to the corre- sponding datasets. However, the limitations of the Graph-to- Text input method for LLMs constrain the model's improvement. Soft prompt suffers from severe misalignment issues, and in some cases even introduces negative effects for smaller models like 3B, with performance drops ranging from 8.6 24.2 percentage point over zero-shot performance. By comparison, our approach shows con- sistent performance improvement across different model scales, bridging the gap between LLMs textual reasoning and topology understanding using one single token effectively and efficiently.
5.3 ABLATION STUDY (RQ2)
How do structural tokens impact the effectiveness of the model? First of all, Figure 2 shows a toy example the performance variation across the selection of different struc- tural token <SOG,> for one specific graph instance. The label of this graph is "True," and plot the generation probability of the LLM outputting 'True' when different structural to- kens are input. When our model selects the expected <500157>, the probability of out- putting the correct answer is maximized. In contrast, other structural token choices lead to highly divergent results, indicating that the structural tokens we generate are highly se- lective, and selecting the appropriate one can significantly enhance the model's performance. Furthermore, we conduct the ablation study with three differ- ent variants as shown in Table 2 (1) w/o Structural Token: Removing the structural token <SOG>> from LLM input and keep only task prompt and textual graph attributes. (2) Ran-
Normaized Ieelood token True'
<S05
0.8
dom Structural Token: Replace the specific <SOG,> with random one. (3) Static Structural Token: Input a static struc- tural token without discrimination. We observe that any vari- ants all lead to a performance drop, confirming that structural tokens play crucial role in guiding LLMs understand graph topology. The statistical indicators demonstrate the highly selective nature of structural tokens: only the correct token generated by our topology-aware graph structural tokenizer is able to faithfully convey the correct topology information of the corresponding graph and thereby yield consistent improve- ments in downstream tasks, whereas random or static substi- tutions fail provide meaningful guidance. This observation provides strong evidence that our LLM has learned to asso- ciate the specific topology-awan token with the correspond- ing graph structures. Consequently, only the properly mapped token can reliably transmit graph topology and drive accurate predictions.
0.6
0.4
0.0
50
100
150
200
250
Structure Token
Figure 2: toy example perfor- mance variation across the selec- tion of different structural tokens. More examples are shown in Fig- ure in Appendix C
How does the use of different hybrid QA methods for instruction tuning affect the perfor- mance of the model? Figure shows the model performance of instruction fine-tuning using three
7

---

Published as a conference paper at ICLR 2026
Abiation Study on Hybrid QAs (LLaMA3-38)
Ablation Study on Hybrid QÁs (LLaMA2-78)
Matching Only
Hybrii QAs
Matching Only
Hybrid
100
KNN Matching - TIF Judgement
100
KNN Matching Tif judgement
AUC RDC
70-
60
60
50
Tox21
ClinTox
BACE
869P
OfinTox
Figure 3: Ablation of tuning the L.LM with different hybrid QAs corpora on five datasets.
different QA corpora. find that the model generally performs better when using all hybrid QAs, particularly with Tox21 achieving remarkable results on the 3B model. At the same time, we observe that fine-tuning with description-token matching occasionally leads to a slight decrease in perfor- mance. This is because it requires the LLM to perform fine- -eeiira structural characterization, which increases the learning difficulty of the model in certain scenarios.
5.4 CASE STUDY & MODEL ANALYSIS (RQ3)
Does the explicit structural token have interpretabil- ity? To verify this, select the first 50 structural to- kens, and as shown in Figure 4 we present the heatmap of their correlation matrix. We observe that the diago- nal entries are 1.0, and beyond trivial self-similarity, the matrix is dominated by relatively weak (around 0.6-0.7)
1.0
10
0.8
Token Index
Correlation
20
0.6
off off-diagonal entries; furthermore, some token (e.g. to- ken <SOG11>> exhibits uniformly low correlation with all others. This pattern indicates low redundancy among <Scg,>s and suggests compact, eetr-orthogonal tok- enization of graph topology.
0.4
0.2
40
0
10
20
30
40
Token Index
Does the selection of structure tokens share similar structural information across datasets? We examine the selection of structure tokens and their correspond- ing structures in different molecule datasets, as shown in Figure 5 Each row represents a set examples which the corresponding structure token selected from different datasets. We find that these molecular structures share the same Be- mis mis-Murcko scaffold (first column), which was extracted using the RDKit open-source toolkit It reveals that molecules sharing the scaffold consistently mapped to the same structural token. This scaffold-leve consistency confirms that <SOG > can capture core topological motifs, thereby providing a concise and non non-redundant compression graph structure.
Figure 4: Correlation heatmap of first 50 structural tokens.
https://www.rdkit..rggg
Do structure tokens and text tokens share the same embedding space? Here we compare the relationship of structural tokens, text tokens, and graph soft prompts of LLaMA2-7BB We randomly sample 300 graphs and gen- erate their soft prompts via the GCN, then extract embed- dings for both structural tokens and frequent text tokens from the LLM's vocabulary. We project all three ones into 2D using PCA followed by t-SNE. Figure 6) visualizes that soft prompts and text tokens occupy clearly separable regions, revealing severe misalignment between graph- derived and Lanuuage-derived embeddings. In contrast, structural tokens are embedded within the neighborhood of text tokens, effectively bridging the graph and lan- guage spaces. This pattern suggests that structural tokens uniformly inject the topology knowledge in the LLM's to- ken space, thereby enabling LLM to read and reason over graph structural information.
Text Embedding Structure Token Embedding Soft Prompt Embedding
100
50
50
100
Figure 6: t-SNE visualization of struc- tural tokens, text tokens, and graph- derived soft prompt embeddings
8

---

Published as a conference paper at ICLR 2026
Scaffold
Molecule #1
Molecule #2
Molecule
Structural Token: <50g223?
Clinn 56
Tox21: # 197
HBBB. a 62
Structural Takan: 50G127
ClinTox:
BBBP:
4021, Tox21:
S S=RNICCIJNICCINICCO
Structural Takan: <50Gss?
ClinToo::
372
clco(ND0oown2eell
Figure 5: Molecules sharing the same scaffold consistently select the same structural token. More examples are shown in Figure 10 Figure II Appendix
5.5 EXTENSION TO NODE-LEVEL CLASSIFICATION (RQ4)
In addition to the graph-level task, our method is easy to extend node-Ievel clas- sification. Besides the global virtual node, each node the graph also assigned corresponding structural token. Thus, we inject the node-level structural token to the LLM, and compare its performance two widely-adopted benchmarks, Cora and Pubmed (Hu et al.) 2020a: Li al. 2024c). The detailed operation please refer to Ap- pendix B.3 A series of baselines are com- pared for comprehensive comparison in- cluding non-LLM Lon approaches (DGI Velickk- ovic et al. 2019), GraphMAE (Hou et al.] 2)22 generalT..M. with extensive param-
Table 3: Performance comparison on node classifica- tion, where bold indicates the best performance and underlined indicates the second-best.
Cora
Pubmed
Model
Acc (f)
F1 (f)
Acc (f)
FI (f)
DGI
17.50
12.44
44.88
38.72
GraphMAE
27.08
23.66
22.03
15.65
LLaMA3 (70B)
67.99
68.05
77.00
64.18
GPT-3.5-turbo
65.67
63.22
75.99
GPT-40
68.62
68.49
77.96
71.79
DeepSeck-chat
65.62
65.77
79.23
74.30
Emb w/ NA
63.59
58.23
74.66
73.15
OFA
23.11
23.30
46.60
35.04
ZEROG
62.52
57.53
79.08
77.94
GraphGPT
24.90
7.98
39.85
20.07
eters (LLaaA3-770B (Dubey et al. 2024), GPT-3.5-turbo (Ouyang et al. 2022). GPT-
Ours (LLaMA3-3B)
91.58
78.62
97.46
85,50
Ours (ILaMA2-7B)
88.80
71.28
96.27
89.64
4o (Hurst et al. 2024), DeepSeek-chat (Guo)
et al. 2025)) and graph specific LLM methods (Emb w/ NA (Li et al. 2024c),
OFA
(Liu et al.
2024 ZEROG (Li et al. 2024b), GraphGPT (Tang al. 2024
As shown in Table 3] our method consistently achieves state-of-the-art performance across all datasets. Remarkably, the proposed <S0G,> with only 3B parameters surpasses much larger mod- els such as GPT-4o and DeepSeek-chat, yielding gains 22.96 percentage point over GPT-4o and 25.96 percentage point over DeepSeek-chat in accuracy. Compared with ZEROG, which shows good performance among GraphLLM baselines, our approach achieves 23% gain in accuracy and 15% improvement in FI. The node-level experiment demonstrates that structural tokens exhibit both global and local structure perspective, due to our task-agnosti alignment strategy.
5.6 HYPERPARAMETER & DESIGN SENSITIVITY ANALYSIS (RQ5)
We conduct structural hyperparameter studies to examine how key design choices affect the effec- tiveness of <SOG>>, First, we evaluate different vocabulary sizes (e.g., 64, 128, 256, 512) and results shown in Table 4 We observe that K=256 achieves the best overall performance. Smaller values fail to adequately distinguish diverse structural patterns, while excessively large values tend overfit and reduce generalization; thus, K=256 provides favourable balance between expres-

---

Published as a conference paper at ICLR 2026
Table 5: Comparison of Anchor Node Selection Strategies.
Task
random
pagerank
betweeness
degrec*
BBBP-p_m
74.4
82,2
80.9
76.9
HIV_HIV WW_.IV_Vatti Lactive
72.0
75.4
62.3
79.7
BACE.Clas
63.5
64.3
65.5
98.4
ClinTox FOADTOX.FDLLPPROEEDD
92.4
99.6
98.0
77.8
Tox21 NR-AR
69.6
59.6
69.7
76.4
Tox21 -RR-AR-LBD
66.6
58.7
64.0
76.0
Tox21 SR-p53
54.6
56.4
63.3
avg.
72.2
70.6
71.0
78.4
siveness and robustness. Our choice of K=256 provides the most appropriate capacity model the structural diversity exhibited in our benchmarks.
In addition, conduct sensitivity analysis anchor node selection. We adopt node degree (*) by default, and compare it with alternatives, i.e., PageRank (Yang et al. 2024b). Betweenness Centrality (Xiang et al. (0233) and random selection, assess their impact on struc- tural encoding and downstream performance, as shown Table 5] The degree-based strategy remains the most stable and effective in most benchmarks. PageRank of- fers slight improvements on BBBP and ClinTTx due to its stronger capture of global centrality in sparse molecular graphs, while random selection often shifts the structural center unpredictably, resulting in unstable encoding and degraded performance.
Table 4: Effect of Structural Vocabulary Size K on Model Performance.
Structural Vocab Size
Dataset
64
128
256
512
BBBP
84.6
71.8
76.9
96.2
ClinTox
91.8
90.1
76.9
90
HIV
52.7
51.9
79.7
55.9
BACE
69.8
62.3
98.4
85.7
Tox21
52.6
53.4
81.6
59.8
avg.
60.1
59.3
81.7
66.8
CONCLUSION
In this paper, propose utilize structural token <SOGg> for LLM explicit topology com- prehension both concisely and accurately. Our method addresses critical issues token inefficiency and cross-modal misalignment of the two mainstream paradigms LLM structural understanding. We empirically validate the effectiveness of <SOG>> on both graph-level and node-level bench- marks, demonstrating that our structural tokens are only well aligned with the LLM's native token space, but also highly selective in capturing graph distinct structural patterns. In the future, we will focus on elaborating its capabilities on more diverse graph data, such dynamic graphs and heterogeneous graphs.
ACKNOWLEDGMENTS
The authors of this paper were supported by National Natural Science Foundation of China (No. T2221102, 92579211 1, 62272301, 623B2071, 62525209), Postdoctoral Fellowship Program of CPSF under Grant Number No. G7B20250806 and the Al for Science Seed Program of Shanghai Jiao Tong University (project number 2025A14S-QY0I).
REFERENCES
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neu- ral Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, December 6-12, 2020, virtual, 2020.
10

---

Published as a conference paper at ICLR 2026
Yukun Cao, Shuo Han, Zengyi Gao, Zezhong Ding, Xike Xie, and S. Kevin Zhou. Graphinsight: Unlocking insights in large language models for graph structure understanding. In Proceedings of the 63rd Anmual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), ACL. 2025, Vienna, Austria, July 27- August 1, 2025, pp. 12096-12134 Association for Computational Linguistics, 2025.
Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, and Yang Yang- Graphllm: Boosting graph reasoning ability of large language model. CoRR, abs/2310.05845, 2023.
Runjin Chen, Tong Zhao, Ajay Kumar Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant. In Forty-first International Conference on Machine Learning, ICML, 2024, Vienna, Austria, July 2 21-27, 2024. OpenReview.net, 2024.
Julian Coda-Forno, Marcel Binz, Zeynep Akata, Matt M. Botvinick, Jane X. Wang, and Eric Schulz. Pheta-in-conters learning in large language models. Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina I Lobanova, Emily Dinan, Fric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Nail, Grégoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. The llama 3 herd of models. CORR, abs/2407.21783. 2024.
Taoran Fang, Yunchao Zhang. Yang Yang, Chunping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, IA, USA, December 10- 16, 2023, 2023.
Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. Talk like a graph: Encoding graphs for large language models. In The Twelfth International Conference Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
Zhong Guan, Hongke Zhao, Likang Wu, Ming He, and Jianpin Fan. Langtopo: Aligning language descriptions of graphs with tokenized topological modeling. CORR, abs/2406. 13250, 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma. Xiao Bi, Xiaokang Zhang. Xingkai Yu. Yu Wu, Z. F. Wu. Zhibin Gou, Zhi- hong Shao, Zhuoshu Li, Ziyi Gao. Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Hanwei Xu, Honghui Ding, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jingchang Chen, Jingyang Yuan, Jinhao Tu, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaichao You, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Meng Li. Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi
11

---

Published as a conference paper at ICLR 2026
Ge. Ruisong Zhang, Ruizhe Pan, Runji Wang. R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang. W. I., Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie. Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang. Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang. Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong. Ying He, Yishi Piao, Yisong Wang. Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yonggiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong. Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang, Yaohui Li, Yi 7heng, Yuchen Zhu, Yunx- ian Ma, Ying Tang. Yukun Zha, Yuting Yan, Z. Z. Ren, Zchui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma. Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song. Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081) 6:633663, September 2025.
Taicheng Guo, Kehan Guo, Bozhao Nan, Zhenwen Liang. Zhichun Guo, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. What can large language models do in chemistry? A comprehen- sive benchmark on eight tasks. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10- 16, 2023, 2023.
Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Har- nessing explanations: LIm-to-lm interpreter for enhanced text-attributed graph representation learning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024a.
Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V. Chawla, Thomas Laurent, Yann LeCun, Xavier Bres- son, and Bryan Hooi. G-retriever: Retrieval-augmennte generation for textual graph understand- ing and question answering. In Advances in Neural Information Processing Systems 38: An- nual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10- 10-17 15, 2024, 2024b.
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. In KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 18, 2022, pp. 594-604. ACM, 2022.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Inter- national Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Linmei Hu, Zeyi Liu, Ziwang Zhao, Lei Hou, Liqiang Nie, and Juanzi Li. A survey of knowledge enhanced pre-trained language models. IEEE Trans. Knowl. Data Eng., 36(4):1413-1440, 2024.
Weihua Hu, Matthias Fey, Marinka Zitikk Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrlPS 2020, December 6-12, 2020, virtual, 2020a.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In 8th International Conference on Learming Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- view.net, 2020b.
Xuanwen Huang, Kaiqiao Han, Yang Yang, Dezheng Bao, Quanjin Tao, Ziwei Chai, and Qi Zhu. Can GNN be good adapter for llms? In Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024, pp. 893-9999990 904. ACM, 2024.
12

---

Published as a conference paper at ICLR 2026
Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker- Whitcomb, Alex Beutel, Alex Borzunov, Alex Cameey, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Ka- mali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codis- poti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Back, An- gela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang. Benjamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng. Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg. Brydon Eastman, Camillo Lugaresi, Carroll I.. Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Chamning Con- ger, Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czamecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, and Dane Sherburn. Gpt-4o system card. CoRR,
2024. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
Yuchen Lei, Yuexin Xiang, Qin Wang. Rafael Dowsley, Tsz Hon Yuen, and Jiangshan Yu. Large language models for cryptocurrency transaction analysis: A bitcoin case study. CoRR, abs/2501.18155 2025.
Xin Li, Weize Chen, Qizhi Chu, Haopeng Li, Zhaojun Sun, Ran Li, Chen Qian, Yiwei Wei, Chuan Shi, Zhiyuan Liu, Maosong Sun, and Cheng Yang. Can large language models analyze graphs like professionals? A benchmark, datasets and models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024a.
Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li. Zerog: Investigating cross-dataset zero-shot transferability in graphs In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024, pp- 1725-1735. ACM, 2024b.
Yuhan Li, Peisong Wang, Xiao Zhu, Aochuan Chen, Haiyun Jiang, Deng Cai, Wai Kin (Victor) Chan, and Jia Li. Glbench: A comprehensive benchmark for graph with large language mod- els. In Advances in Neural information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10- 15, 2024, 2024c.
Zhixun Li, Xin Sun, Yifan Luo, Yanqiao Zhu, Dingshuo Chen, Yingtao Luo, Xiangxin Zhou, Qiang Liu, Shu Wu, Liang Wang, and Jeffrey Xu Yu. GSLB: the graph structure learning benchmark. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Informa- tion Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
Zhucong Li, Jin Xiao, Bowei Zhang, Zhijian Zhou, Qianyu He, Fenglei Cao, Jiaqing Liang, and Yuan Qi. Chemhts: Hierarchical tool stacking for enhancing chemical agents. CoRR, abs/2502. 14327, 2025.
Qika Lin, Tianzhe Zhao, Kai He, Zhen Peng, Fangzhi Xu, Ling Huang, Jingying Ma, and Mengling Feng. supfrvuue quantized representation for seamlessly integrating knowledge graphs with large language models. In Proceedings of the 63rd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 August 1, 2025, pp. 13587-13602. Association for Computational Linguistics, 2025.
13

---

Published as a conference paper at ICLR 2026
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In The Twelfth Inter- national Conference Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
Jiawei Liu, Cheng Yang. Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S. Yu, and Chuan Shi. Graph foundation models: Concepts, opportunities and challenges. IEEE Trans. Pattern Anal. Mach. Intell., 50255:0033,5044, 2025a.
Zequn Liu, Wei Zhang, Yingee Xia, Lijun Wu, Shufang Xie, Tao Qin, Ming Zhang, and Tie- Yan Liu. Molxpt: Wrapping molecules with text for generative pre-training. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 1606-1616. Association for Computational Linguistics, 2023a.
Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Molca: Molecular graph-language modeling with cross-modal projector and uni-modal adapter. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore. December 6-10, 2023, pp. ciation for Computational Linguistics, 2023b.
Asso-
Zipeng Liu, Likang Wu, Ming He, Zhong Guan, Hongke Zhao, and Nan Feng. Multi-view empow- ered structural graph wordification for language models. In AAAI-25, Sponsored by the Associ- ation for the Advancement of Artificial Intelligence, February 25- March 2025, Philadelphia, PA, USA, pp. 24714-2472 AAAI Press, 2025b.
lya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Harry L Morgan. The generation of a unique machine description for chemical structures-a tech- nique developed chemical abstracts service. Journal of chemical documentation, 5(2):107-113, 1965.
OpenAI. GPT-4 technical report. CORR, abs/2303.08774, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 35: Annual Conference Neural Informa- tion Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Ed- ward Z. Yang. Zachary Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 8024 8035, 2019.
Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Seyed Mehran Kazemi, Rami Al- Rfou, and Jonathan Halcrow. Let your graph do the talking: Encoding structured data for 1lms. CoRR, abs/2402.05862, 2024.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. EMNLP. IJCNLP 2019, Hong Kong. China, November 3-7, 2019, pp. 3980-39909999 39900. Association for Compu- tational Linguisties, 2019.
14

---

Published as a conference paper at ICLR 2026
Jiabin Tang. Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024, pp. 491-500. ACM, 2024.
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: large language model for science. CoRR, abs/221 1.09085, 2022.
Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V. Chawla, and Panpan Xu. Graph neural prompting with large language models. In Thirty-Eightn AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Appli- cations of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pp. 19080-19088. AAAI Press, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao. Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Ilyaan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodrigue7, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307.092$$ 2023.
Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liô, Yoshua Bengio, and Devon Hjelm. Deep graph infomax. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Duo Wang, Yuan Zuo, Fengzhi Li, and Junjie Wu. Llms as zero-shot graph learners: Alignment of GNN representations with LLM token embeddings. In Advances in Neural Information Process- ing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024a.
Heng Wang, Shangbin Feng, Tianxing He, Zhaoxuan Tan, Xiaochuang Han, and Yulia Tsvetkov. Can language models solve graph problems in natural language? In Advances in Neural Informa- tion Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
Jianing Wang, Junda Wu, Yupeng Hou, Yao Liu, Ming Gao, and Julian J. McAuley. Instructgraph: Boosting large language models via graph-centric instruction tuning and preference alignment. In Findings of the Association for Computational Linguistics. ACL 2024, Bangkok, Thailand and vir- tual meeting, August 11-16, 2024, pp. 13492-135115 Association for Computational Linguistics, 2024b.
Runze Wang, Mingqi Yang, and Yanming Shen. Bridging molecular graphs and large language models. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25- March 2025, Philadelphia, PA, USA, pp. 21234-21242. AAAI Press, 2025a.
Zhuang Wang, Linna Zhou, Xuekai Chen, Zhili Zhou, and Zhongliang Yang. STLC-KG: A social text steganalysis method combining large-scale language models and common-sense knowledge graphs. AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25- March 4, 2025, Philadelphia, PA, USA, pp. 25461-25469. AAAI Press, 2025b.
Zhenqin WU, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay S. Pande. Moleculenet: A benchmark for molecular machine learning. CoRR, abs/1703.00054, 2017.
15

---

Published as a conference paper at ICLR 2026
Nan Xiang, Qilin Wang, and Mingwei You. Estimation and update of betweenness centrality with progressive algorithm and shortest paths approximation. Scientific Reports, 13(1):17110, 2023.
Ling Yang, Ye Tian, Minkai Xu, Zhongyi Liu, Shenda Hong, Wei Qu, Wentao Zhang, Bin Cui, Muhan Zhang, and Jure Leskovec. Vqgraph: Rethinking graph representation space for bridging gnns and mips. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, OpenReview.net, 2024a.
Mingji Yang, Hanzhi Wang, Zhewei Wei, Sibo Wang, and Ji-Rong Wen. Efficient algorithms for personalized pagerank computation: A survey. IEEE Trans. Knowl. Data Eng., 36(9):4582-460) 2024b.
Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang. Language all graph needs. In Findings of the Association for Computational Linguisties: EACI. 2024, St. Julian's, Malta, March 17-22, 2024, pp. 1955-1973. Association for Computational Linguisties, 2024.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. In Advances in Neural Information Processing Systems 33: Annual Conference Neural Information Processing Systems 2020, NeurlPS 2020, Decem- ber 6-12, 2020, virtual, 2020.
Haiteng Zhao, Shengchao Liu, Chang Ma, Hanman Xu, Jie Fu, Zhihong Deng, Lingpeng Kong, and Qi Liu. GIMLET: A unified graph-text model for instruction-based molecule zero-shot learn- ing. In Advances in Neural Information Processing Systems 36: Anmual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023a.
Jianan Zhao, Le Zhuo, Yikang Shen, Meng Qu Kai Liu, Michael M. Bronstein, Zhaocheng Zhu, and Jian Tang. Graphtext: Graph reasoning text space. CoRR, abs/2310.01089, 2023b.
Zihan Zhao, Da Ma, Lu Chen, Liangtai Sun, Zihao Li, Yi Xia, Chen, Hongshen Xu, Zichen Zhu, Su Zhu, et al. Developing chemdfm as a large language foundation model for chemistry. Cell Reports Physical Science, 6(4), 2025.
A ADDITIONAL DETAILS ON EXPERIMENT SETUP
A.1 DATASETS
MoleculeNet is benchmark suite designed to standardize datasets and create consistent evalu- ation framework for machine learning models in computational chemistry (Wu et al., 2017). The following five datasets (BBBP, BACE, HIV, CinTox, Tox21) all come from this benchmark. We follow the dataset selection and splitting criteria specified in GIMLET (Zhao et al. 2023a). As for downstream tasks, we divide the datasets into a ratio of 8:1:1 and report the results on the test sets. There are single-task and multi-task setups, and multiple tasks from the same dataset differ only in task labels while sharing identical molecules. Data statisties shown in Table 6)
BBBP The BBBP Blood-Braii Barrier Permeability) dataset is part of the MoleculeNet collec- tion. In particular, it is used to predict whether given chemical compound can permeate the blood- brain barrier (BBB). BBBP contains binary labels for compounds that indicate whether they can cross the blood-braie barrier or not, for compounds that can pass through the BBB and 0 for those that cannot. These compounds are represented by SMILES expressions. The dataset includes 2,039 compounds with various chemical features, including molecular properties such atom type, bond type, molecular weight, and so on. However, they are not used as assistant information in the model input in this task. The dataset is widely used for drug development research. This is important in the design of drugs aimed at treating brain-related diseases, as only compounds that can cross the BBB able have an effect the central nervous system.
16

---

Published as a conference paper at ICLR 2026
Table 6: Graph-level classification datasets statistics.
# total samples
positive prop. ())
negative prop. %)
dataset
train
valid
test
train
valid
test
train
valid
test
BACE Class
1210
42.6
55.6
60.5
57.4
44.4
39.5
BBBP up
1631
204
204
82.2
54.9
52.5
17.8
45.1
47.6
Clin'Txx CT TOX
1184
148
148
8.0
4.7
6.8
92.0
95.3
93.2
ClinTox FDA APPROVED
1184
148
148
93.3
96.0
93.9
6.7
4.1
6.1
HIV active
32896
4112
3.8
3.2
96.3
96.8
Tox21 NR-AR
6258
783
4.0
4.0
3.5
96.0
96.0
96.6
Tox21 NR-AR-LBD
6258
782
783
3.1
3.2
2.4
96.9
96.8
97.6
Tox21 NRRAhR
6258
782
783
9.4
11.1
11.8
90.6
88.9
88.3
Tox21 NR-Aromatase
6258
782
783
3.3
5.8
6.0
96.7
94.3
94.0
Tox21 NR-ER
6258
783
10.3
8.9
89.7
90.4
91.1
Tox21 Tox21_NREEKS NR-ER-LBD
6258
782
783
4.8
3.7
2.7
95.2
96.3
97.3
Tox21 Tox21_NR-PP NR--PPPP R-gamaa
6258
782
783
2.1
4.1
2.8
97.9
95.9
97.2
Tox21 SR-ARE
6258
782
783
11.5
13.6
15.1
88.5
86.5
84,9
6258
783
3.1
4.5
4.2
96.9
95.5
95.8
Tox21_SR SR--HS
6258
782
783
4.5
5.6
6.0
95.5
94.4
94.0
Tox21 SR-MMP
6258
782
783
11.4
14.2
12.3
88.6
85,8
87.7
Tox21 SR-p53
6258
782
783
4.4
9.6
9.2
95.6
90.4
90.8
BACE The BACE dataset from MoleculeNet designed for binary classification, aiming to pre- dict whether a molecule will bind to the human /-secretase, which is an aspartic acid protease important in the formation of myelin sheaths in peripheral nerve cells. In the BACE binary classifi- cation task, label of I indicates that the molecule is BACE inhibitor, while label of 0 signifies that the molecule is not a BACE inhibitor. The dataset contains 1513 molecules, each with molecular features, i.e. their SMILES expressions. It is widely used for evaluating machine learning models in drug discovery.
HIV The HIV dataset in MoleculeNet aims to predict whether drug can inhibit HIV replication. This dataset consists of 41 120 molecules with features e.g. smile expressions and their correspond- ing activity against the HIV virus. The task is binary classification, label of 1 indicates that given drug molecule can inhibit HIV replication, while a label of 0 signifies that can not. The dataset widely used for evaluating machine learning models in the context of drug discovery and molecular bioinformatics, particularly the realm of HIV research and drug development.
The ClinTox dataset is derived from drug discovery studies and collected in Molecu- leNet. It focuses on clinical trial outcomes and drug safety evaluation, containing 1480 compounds annotated with two binary classification tasks. One task, FDA-APPROVED. predicts whether a drug molecule is approved by the U.S. Food and Drug Administration (FDA), where label denotes FDA- approved drugs and 0 denotes non-approved drugs. The other task, CT-TOX, determines whether a drug molecule has failed clinical trials due to toxicity, with for failure and for success and continuation. This dataset is widely used for toxicity prediction and drug safety assessment.
Clin'Tx
Tox21 The Tox21 dataset originates from the Toxicology in the 21 Century (Tox21) program and is included in MoleculeNet. It is designed for evaluating the human toxicity of chemical compounds across 12 biological targets. It consists 7,823 molecules, each annotated with 12 binary labels indicating whether the compound activates or inhibits specific target. Specifically, the nuclear re- ceptor (NR) tasks include NR-AR, NR-AR-LBD, NR-AhR, NR-Aromatase, NR-ER, NR-ER-LBD, and NR-PPAR-gamen the corresponding receptor. The stress response (SR) tasks include SR-ARE, SR-ATADS, SR-HSE, SR-MMP, and SR-p53, which assess whether compound induces responses such antioxidant signaling. DNA damage, heat shock, mitochondrial dysfunction, or p53-mediated stress pathways. All tasks formulated binary classification problems, where denotes activation and 0 denotes no activation. This dataset has become a widely adopted benchmark in computational toxicology and drug safety evaluation.
each requiring the prediction of whether molecule activates or inhibits
17

---

Published as a conference paper at ICLR 2026
A.2 EVALUATION METRICS
To quantitatively evaluate the graph classification results, we use AUC-ROC (Area Under the Re- ceiver Operating Characteristic Curve) assess the performance of binary graph classification tasks, as most tasks exhibit a significant class imbalance. AUC-ROC measures the model's ability to dis- tinguish between classes by plotting the true positive rate (TPR) against the false positive rate (FPR) across different thresholds. The higher the value, the better the model's discrimination ability.
For the calculation of the AUC-ROC metric in practice, the maximum output token length is lim- ited to 100. To generate output, the LLM sampling parameters are set with temperature of 0.6 which strikes a balance between randomness and coherence, while limiting the token selection those with cumulative probability mass of 90%. Then we parse the generated text to extract answers. Specif- ically, we perform matching by defining a positive set (eg. "eg.l""yes"."tree" negative set (eg.[" ("oo", "false", "inactive", "rejected", "not approved"].. If any word/phrase from the positive set appears in the generated text, the prediction is assigned I; otherwise, if any one from the negative set appears, it is assigned O. To mitigate substring conflicts (e.g., positive phrase em- bedded in negations), we enforce matching precedence: negatives are evaluated first, and positives considered only in the absence of negative match. The resulting predictions paired with ground-truth labels (0/1) to compute AUC-ROC.
"active", "approved")) and
For multi-task evaluation, the reported mean is obtained by averaging the mean performance across individual tasks. Assuming independence between tasks, the overall standard deviation is derived as the square root of the mean of squared per-task standard deviations.
To quantitatively evaluate the node classification results, we adopt accuracy and Fi-score as stan- dardized metries. Accuracy calculated the ratio of correctly classified nodes the total number of nodes, providing a simple measure of overall classification performance, given by:
N ii=
Iw: (i)
Accuracy
N
where /wu = (.) an indicator function that equals the true label y, matches the predicted label 9r, and 0 otherwise, and N is the total number of nodes. However, accuracy may not be sufficient when dealing with classification tasks. Therefore, we also use Fl-score, which is the harmonic mean of precision and recall. Precision measures the proportion of true positive predictions among all positive predictions, while recall evaluates the proportion of true positives identified from all actual positives. The Fl-score balances both metries, offering more reliable performance measure. Given the confusion-matrix counts (TP, FP, TN, FN) Where TP is the number of true positives, FP is the number of false positives, TN is the number of true negatives, and FN is the number of false negatives. The F1-score for each binary task is calculated as:
TP
TP
Fi-score = 2 Precision x Recall
Precision
Recall
TP FP'
TP + FN'
Precision + Recall
In multiclass settings, the micro-averaged Fl-score aggregates counts over all classes when comput- ing precision and recall. Let TPx, FPx, and FN, denote the true positives, false positives, and false negatives for class k. Define
TPR TP
En TP
Precisionmiero
Recallmiero
CT + FP,)
ETPTAA+FR
and
2 Precisionmiiro Recallmiero
Fl-scoremicro
Precisionmiiro + Recallmiero
Micro-FI reflects overall, sample frequency weighted performance. In this paper, we use micro-F1 for all Fl F1-score evaluations.
For the calculation of accuracy in practice, we also set the maximum output token length limited to 100. We count a prediction as correct if the ground-truth label appears in the generated output, and report the proportion of correct predictions over the total number of samples. For F1, we extract a predicted label by scanning the output for the first match (falling back to unknown if none is found), then compute micro-F1 over multiple classes based on predicted labels and ground truths. We report the mean accuracy and F1-score of our method across three runs.
18

---

Published as a conference paper at ICLR 2026
A.3 BASELINES
Baselines for Graph-level Classification We incorporate both non-LLM-based models and LLM- based models as baselines for graph classification tasks. The non-L.LMbbased models include graph prompt learning method GPF (Fang et al. (023) with AttrMasking (Hu et al. 2020b) and GCL YYou al. 2020) as pretrained GNN models, and graph-language tuning method MolCA (Liu et al, 2023b) and its variant MolCA-GS. For the LLM-based models, we categorize the approaches as follows.
Zero-shot Inference: Directly input textual graph attributes and instruction prompts into LLMs, utilizing their pre-trained knowledge and reasoning capabilities (Hu et al. 2024).
Few-shot Learning: Directly input textual graph attributes, instruction prompts as well as amples with complete textual attributes and ground truth labels into LLMs, leveraging LLMs' in context learning capabiliiies (Coda-Forno et al. 2023), which means the few shot examples can guide LLMs' understanding of the task. The few shot examples sampling strategy include random sampling and selecting similar samples based on Morgan similarity (Morgan 1965). In our settings, we use 8-shot learning (Guo et al. 2023) with 4 samples for each category.
Supervised Fine-Tuning: Apply supervised fine-tuning via Low-Rank Adaptation (Hu et al. 2022) adapt the model to downstream tasks. In our setup, we combine data from all tasks to form the training set, and use the jointly trained [.LM to evaluate the test split of each task.
Soft-Prompt: Employ a projector, i.e.a two-layer Multi-Layer Perceptron (MIP) with ReLU activation, to map graph embeddings into the LLM space, training only the MLP while keeping the LLM frozen. We use ErossEntropyLoss by comparing the LLM's predicted logit for the token "True" with the true labels (0/1), applied to the first generated token.
Baselines for Node-level Classification We evaluate node classification against three baseline families: (i) graph self-supervised learning (SSL) methods that learn transferable structural repre-
sentations (Li et al. 2202)), exemplified by DGI (Velickovic et al. (211)) and GraphMAE (Hou et al.) 2 (ii) LL baselines that rely solely on semantic information, like LLaMA3-70B (Dubey et al. 2024). GPT-3.5-turbe (Ouyang et al. 2022), GPT-4o (Hurst et al.) 2024). and DeepSeek- chat (uuo et al. 2025): (iii) LLM- -raph integration frameworks that leverage structural and se- mantic information within ALM-based pipelines for node classification. Emb w/ NA is a simple training-free baseline that combines structural and semantic cues for zero-shot inference (Li et al. 2024c). We also include two advanced integration frameworks: OFA (Liu et al. 2024), which tex- tualizes all nodes and edges into a human-readable form, encodes heterogeneous domains into a shared language space, and adapts to downstream tasks by inserting task-specific prompting sub- structures; and ZeroG (Li et al. 2024b), which encodes node attributes and class descriptions with language model and employs prompt-based subgraph sampling with lightweight fine-tuning to tackle cross-dataset zero-shot transfer. Collectively, these baselines span structure-only. semantics- only, and structure- joint paradigms.
Some baseline results are taken from other works. In Table 1, the AUC-ROC results of GPT-4 come from Zhao al. (2025); the results of GPT-4o, De and LLaMA3-70B come from Li (2025); the results of Galactica-1200
et al.
come from Liu et al. 2023a):: the results of GIMLET Zhao al. (2023a); the results of Gur-Auurasi,nn GPF-GCL, MolCA-S, MoICA-GS
come from
come from
Wang et al. (2025a). In Table 3
the accuracy and F1-score results of all baselines are all
taken from
Lt al. (202cc).
B ADDITIONAL DETAILS ON IMPLEMENTATION
B.1 MODEL DETAILS AND TRAINING HYPERPARAMETERS
We encode newly assigned structural attribute with SentenceTransforme all-MiniLMM-L6-v2 (Reimers & Gurevych, 2019) for initial node features. We employ two-layer GCN as the encoder in structural tokenizer, and set the size of structural vocabulary to 256. We adopt two widely used large language models LLaMA2-7B-chatt (Touvron et al. 2023) and Llama-3 2-.-----Bll --I-ttuutct (Dubey et al. 2024) as our backbones. All models are implemented in Py MoTcch (Paszke et al., 2019). In the structural tokenizer training phase, we firstly pretrain the GCN reconstruction task warm-up with a learning rate of x 10 2. Then we optimize the whole structural tokenizer with two param- eter groups: GCN with learning rate 5 10-2 and structural tokens discrete embeddings 0.5, using
2019)
19

---

Published as a conference paper at ICLR 2026
Adam optimizer (Kingma & Ba 2013). In token alignment with hybrid QAs stage and task specific finetuning stage, we optimize the newly added token embeddings and LLM backbone with learning rate 5 x 10 and LoRA rank of 16, using AdamW optimizer (Loshchilov & Hutter 2019).
B.2 DETAILS DOWNSTREAM APPLICATIONS
After token alignment with hybrid QAs, finetune the embeddings corresponding to the newly added structural tokens and the parameters of LLM backbone for application in downstream tasks. Due to severe class imbalance observed in the training sets of most tasks (shown in (Talle 6), apply various data balancing techniques during finetuning. For some tasks, we also employ joint training with other tasks finetune the L.M.. Specifically, for the LLaMA3-3B model, we apply class balancing with a 1:1 ratio by duplicating minority class samples to match the majority on tasks BBBP p np. Coi FDA APPROVED, ClinTox CT TOX, and HIV HIV active, while re- taining the original training data distribution for BACE Class. Each task is finetuned separately, and evaluation is conducted using the checkpoint from the third epoch. For the Tox21 benchmark, we jointly finetune on the raw training data from all 17 tasks, including data from other four bench- marks, and evaluate the second-epoch checkpoint for Tox21 performance on its 12 subtasks. For the LLaMA2-7B model, we use unbalanced raw training data for
and Tox21_SB HSE, and apply 1:1 balancing ratio as mentioned above for Tox21_NR-AA LBD, Tox2_NR-ER- LBD, Tox21 SR-ARE, ClinTox FDA APPROVED. ClinTox CT TOX, and HIV HIV active. For BBBP p np. Tox21 NR-AR, Tox21 NR-AbR, and Tox21 such that the number of minority class samples is adjusted to one-fifth of the majority class samples, and for BACE_Class, we retain the original distribution. All tasks are finetuned individually, with evaluation based on the second epoch checkpoint, except for TOx21_NR-ER-LI third epoch. For the remaining Tox21 tasks (NR-Aromatase, SR-ATADS, SR-MMP, SR-p53), we jointly finetune on unbalanced raw training data from all 17 tasks, evaluating the third epoch check- point for the four tasks. It's worth noting that we do not make any changes to the validation set and test set distribution.
S---PPARRaaaass we resample the data
which uses the
B.3 EXTENTION TO NODE-LEVEL CLASSIFICATION
To extend our method to node-level tasks, we construct ego- ego-graphs by sampling neighborhood struc- ture for each target node, enabling processing within the standard graph paradigm. For each node, we sample its 2-hop ego-graph, and map it to node-level structural tokens with our topology-aware structural tokenizer trained on graph tasks. We do not need to add global node to the constructed ego-graphs any more. The specific <SOG>> corresponding to the target node is chosen as the struc- tural token to represent the local neighborhood topology of the target node and we insert it in the textual LLM input for the classification of the downstream nodes.
B.4 PROMPT DESIGN OF <509
We incorporate structural tokens into the task prompt and textual graph attributes to construct com- plete LLM input prompt that we can obtain class labels via generative inference. Detailed LLM input prompts for each task are provided in the corresponding boxes (from page 21 to 26).
C ADDITIONAL CASE STUDIES
We provide additional examples illustrating how different structural tokens influence LLM's output probabilities discussed in Section 5.3 as shown in (iiurre 7 Figure Figure 9 Also, we examine more examples of the selection of structural tokens and their corresponding structures in the training split of datasets (instead of test split in Section 5.4). shown in Figure 10 Figure 11
D USE OF LLMS
The core research presented in this paper directly incorporates Large Language Model as an integral component of proposed pipeline, since we are dedicated to improving LLM's graph reasoning capabilitics and mitigating structural hallucinations. Specifically, we add special structural tokens to the LLM input space for more accurate and concise LLM topology understanding. The LLM serves as the main inference model to generate answers in our graph- graph classification and node classification tasks.
evaluations, including
is crucial to clarify that the LLM is used in this operational role and is not involved in the ideation, conceptualization writing of this paper. The research ideas and writing entirely our own.
20

---

Published as a conference paper at ICLR 2026
LLM input prompts for BBBP p np
I of text |> < start header id system< end end header i You are helpful and reliable assistant.< |eot_id|>
< |bbegin_o_teex
user< end_header.id
You are chemistry expert. Classify the given molecule into the correct calegory based on its molecular structure (token) and SMILES expression. Each structure token represents a unique graph pattern (e.g., kind of similar molecular graphs).
[Molecule] smiles waw.eesssrpreesioo... of Jargel molecule
[Structure Token] <SOG>>
[TTss]] Does this molecule have blood-brain barrier permeability (BBB penetration)? True for BBB permeable and False for not BBB permeable. Output the complete correct answer from the following two options:
True
2. False [Answer]< /eoterr</eottiddl >< / ht >LM Assistant> /
LLM input prompts for Tox21_NR-A
begiin_of _teet >> eettart_header.ie system< ( You are a helpful and reliable assistant.< leo__id|>>
Ibegin of text |><| header idl> user< end header idl>
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., kind of similar molecular graphs).
[Molecule] smiles_eeap wet.llessepreesioo... fargel jarget
[Structure Token] <S09
[Task] Does this molecule activate the androgen receptor (NR-AR)? True for active and False for inactive. Output the complete correct answer from the following two options:
True
2. False
[Answer]< /eoterr//eotiidde >< start sttttttt LLM Assistants / /end
LLM input prompts for Mos21_NR-AR--LBD
/begin_of.texx >> www.tt..eaeeeeee.eeeeee
system< \
You are helpful and reliable assistant.< leot_id|>
< <lbegin of text |><| start header idl> user< end header idl>
You chemistry expert. Classify the given molecule into the correct category based its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g.. kind of similar molecular graphs).
[Molecule] smiles wewpsesseeeppeeseo..oo.. darget_mmlecut [Structure Token] <S0gg>>
domain of the androgen receptor (NR- AR-LBD)? True for active and False for inactive. Output the complete correct answer from the
[Task] Does this molecule activate the
following two options:
True
2. False
[Answer]< eot eot
idd >< lstart... lattarttaaaeeee header id >LMM Assistant< lndd header i
21

---

Published as a conference paper at ICLR 2026
LLM input prompts for Mox21NRRAhie
[lbeginnof _text >> <|
system< het.eaddeeaaaeeeeaaaaaaaae.e.e.. /
You are helpful and reliable assistant.< (eot_id]
< lbegin text |>< start hww.heeeeeriic idd user< worddheaderiid
You chemistry expert. Classify the given molecule into the correct category based its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., a kind of similar molecular graphs).
[Molecule] smiles expression.... of garget.molecule [Structure Token] <509g>
[Task] Does this molecule activate the aryl hydrocarbon receptor (NR-AhR)? True for active and False for inactive. Output the complete correct answer from the following two options:
1. True
2. False
[Answer] leot idl >< sstart header id] >LLM Assistant< lnd header i
LLM input prompts for
begiin_of _text >> stastttteadeer..... system< /
You are helpful and reliable assistant. /eot.id]
Ibegin of text start header idl> user< end header idl>
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., kind of similar molecular graphs).
[Molecule] wmmpeeseerrressoon. dargee jarget
[Structure Token] <S09
[Task] Does this molecule inhibit the aromatase enzyme ? True for active and False for inactive. Output the complete correct answer from the following two options:
True
2. False
[Answer]< /eaterr//eottidde >< start idd LLM Assistants hen..eedee..deedddll....
LLM input prompts for NR-ER
NR-ER
begin of text >> < start header id system< end header id
You are helpful and reliable assistant i
|begin_of _text > www.ttthaeerriide
user< lnd_heederridll
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents unique graph pattern (e.g.. a kind similar molecular graphs).
[Molecule] smiles expression of tar get molecule [Structure Token] <SOG>>
['Task] Does this molecule activate the estrogen receptor (NR-ER)? True for active and False for inactive. Output the complete correct answer from the following two options: True
2. False
[Answer]< lect_id >< start www.tt.heade.iiee.. LLM Assistant< header.rdd
22

---

Published as a conference paper at ICLR 2026
LLM input prompts for Tox21 NR-ER-IBD
I of text |> < start header system< end end header i
You are helpful and reliable assistant. |eot_id|>
< |bbeginoo_texx | sttttteeeeddd user< end_header.idl
You are chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., kind of similar molecular graphs). [Molecule] smiles wew.feesserrresiooo... of [Structure Token] <SOG>>
molecule
['Task] Does this molecule activate the ligand-binding domain of the estrogen receptor (NR-ER- LBD)? True for active and False for inactive. Output the complete correct answer from the following two options:
True
2. False [Answer]< /ott >< pt//sttt..eeadr.idl -LLM Assistants hen..eeaee..............
LLM input prompts for Vox2I_NR-PPAR-gamme
beggin_of _teet >> sttsssttttteeeeetiie system< system</ hendde/eee.deeeeidddddd
You are a helpful and reliable assistant. /eot.idd
Ibegin of text staatt header idl> user< end header idl>
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., a kind of similar molecular graphs).
[Molecule] wwwiles_erpressiiieeeeee www.rreessiooooooo.. farget
[Structure Token] <S09
[Task] Does this molecule activate the peroxisome proliferator-actin acttvvate receptor gamma (NR- PPAR-gammal True for active and False for inactive. Output the complete correct answer from the following two options:
1. True 2. False
[Answer]< / /tart header.dd >LM Assistant< hen.ssaatt//edd.ddee..........
LLM input prompts for Tox21
|bbeginno -eett >> < ststttrttii header
startheaderr -id systems / h
You are helpful and reliable assistant leot i
I of text |><|
header id| user< end header idl>
You are a chemistry expert. Classify the given molecule into the correct category based on its
molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., a kind of similar molecular graphs).
[Molecule] www.es_eeppeeeeeeeeeeeeeeepepeee wwwpressiooooooo....e.o www.tattaooleeeeee .molecul [Structure Token] <50g
[Task] Does this molecule activate the antioxidant response element (SR-ARE)? True for active and False for inactive. Output the complete correct answer from the following two options:
True
2. False
[Answer]< lot eot
idl >< lstart. sttattheader: header id >LMM Assistant< end header i
23

---

Published as a conference paper at ICLR 2026
LLM input prompts for S0221SR-ATADD
Abegin_of.text >> <ltsttttteaeeerraaeeee > system< /nd_.eader.dddi h
You are helpful and reliable /eott.iit///
Ibegin_off _text |><|start. header.edrriidi user< end_header.idl
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., a kind of similar molecular graphs).
[Molecule] smiles ofwpessioonoof wtw.oofffaaattttmmeoeeeeee [Structure Token] <SOgE>
[Task] Does this molecule activate ATAD5 signaling (SR-ATAD5)? True for active and False for inactive. Output the complete correct answer from the following two options:
True
2. False httrr//teeld.i.
/
LIM input prompts for Tox21_SR-HST
|bbein_of destintof_textt _teett >> < |lssstttreeeerre...
You are helpful and reliable assistant.< /eot_id|
< <lbegin_of.text |><] header.rd user end_header.id
You chemistry expert. Classify the given molecule into the correct category based its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., kind of similar molecular graphs). [Molecule] petaieeseepreesioo... [Structure Token] <SOG>>
detrget_molecne
[Task] Does this molecule activate the heat shock element (SR-HSE)? True for active and False for inactive. Output the complete correct answer from the following two options:
1. True False
[Answer]< lect_id| >< | LLM Assistant< Jenbsmnek/ed.
LLM input prompts for Tox21 SR-MMP
|beein_of www.oftett////ooooooooooooo///ee < start ietterrrheeddrrdd systems | enddheedeer_id
You are helpful and reliable assistant, eeo__id
|bbeginnof wtett stclsttttheheee...
user< end_header..ic
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents unique graph pattern (e.g., a kind similar molecular graphs).
[Molecule] smiles_eerp cetttcsserrreesioooooo jargel_mmleeull [Structure Token] <S0Gg>>
[Task] Does this molecule activate the mitochondrial membrane potential stress response (SR- MMP)? True for active and False for inactive. Output the complete correct answer from the following two options:
1. True
False
[Answer]< lect_idi >< Ittssttteaderrr... Assistants hend.header.didd
24

---

Published as a conference paper at ICLR 2026
LLM input prompts for Tox21_Shh SRRp53
system< You are helpful and reliable assistant.< eeot >
[lbegin_of _teet >> <| atasstttteeteeeeeetett het.eaddeeeeeeeeeeeeeeeeee...e.e /
< lbegin text |>< start www.heeeerriie lil user< worddheaderiid
You chemistry expert. Classify the given molecule into the correct category based its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., a kind of similar molecular graphs). [Molecule] smiles .ww.es....peessiion..... expression.o. of farget molecule
[Structure Token] <509
[Task] Does this molecule activate the p53 stress response pathway (SR-p53)? True for active and False for inactive. Output the complete correct answer from following two options:
1. True
2. False
[Answer] leot id >< sstart header id >LLM Assistant< lnd header i
LLM input prompts for Clinoox_EDAAAPPROYYY
beggin_of _teet >> stasttteeeeeeriid system< eed_heeddrr_dd
You are helpful and reliable assistant /eot_idd>>
Ibegin of text |><| header idl> user< end header idl>
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents a unique graph pattern (e.g., a kind of similar molecular graphs).
[Molecule] smiles_expre ee..pssssioooooo jarget [Structure Token] <S09
[Task] Has this molecule been approved by the FDA? True for FDA approved and false for non-approved. Output the complete correct answer from the following two options: True
2. False [Answer]< /eottrr//eottiddi ><
star LLM Assistants /end.header.idd
LLM input prompts for ClinTox CT TOX
begin of text >> start header id system< end header id
You are helpful and reliable assistant. /eotiid|>>
lbbeginof _text > starttheaderrrteteee wwwahhradiidll user< lnd_header
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents unique graph pattern (e.g.. a kind similar molecular graphs).
[Molecule] smiles expression of target molecule [Structure Token] <SOg,>
['Task] Is this molecule associated with clinical toxicity? True for clinically toxic and false for non-toxic. Output the complete correct answer from the following two options: True
2. False
[Answer]< /etetl/eeeettae sot__ii >< start wew.artthhaaerric LLM Assistant< lend header.idrr
25

---

Published as a conference paper at ICLR 2026
LLM input prompts for HIV HIV active
|begin_of -tettttofftettttti _text >>
starrtheaeee.iid > systems /end_headr...ddd
You are helpful and reliable assistant.
[begin_of.text >><|start www.er.-haderriddl user< end_header.idl]
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents unique graph pattern (e.g., a kind similar molecular graphs).
[Molecule] smiles__ere exppeessooeessio... garg [Structure Token] <SOG>>
[Task] Does this molecule inhibit HIV replication? True for active molecules which can inhibit HIV and false for inactive ones. Output the complete correct answer from the following two options:
1. True
False
[Answer<<]ectee >< start i Assistant< / lend.header.id
LLM input prompts for BACE Class
libeginnof feeetttttttt/e >> www.stttttteerrrreerr _id system< /en.emeeeedddeeeee.e.
You are helpful and reliable assistant. /eot..d
|bbeginnof _text stallsteaeerrree.eee
user< lnd_headerridl
You are a chemistry expert. Classify the given molecule into the correct category based on its molecular structure (token) and smiles expression. Each structure token represents unique graph pattern (e.g., a kind of similar molecular graphs). [Molecule] smiles expression of target molecule [Structure Token] <S0g,>>
[Task] Is the binding result of the molecular beta-secretase true false? Output the com- plete correct answer from the following two options:
True
2. False
staart.hhade [Answer]< eot. idl >< Istart header idd MLLM Assistant< end header i
Tox21 Tax21_NR-Aromaln MR-Arrmssase: posiive case 180
Tox21 1,21__R_ER posrivee case # 369
Tax21_ SR-ATADS: posiive case A
Nommatzee Bkaihood of taken Toual
Mormallzad Belhhood cood tolion Thua"
<50
<50
<505
0.0
0.6
Normalaad Whellrood
0.6
0.6
0.
0.-
0.0
50
100
150
200
250
100
150
700
250
50
1000
150
700
250
Structure Token
Structure Token ID
Structure Token
Tox21_SR-MMP:
Tox21,
Tox21_ 3R-p53k posilive
Nommltesike thood ofteien Trae
Norralzed Bethood of TTus"
1.0
<50
1.0
<50% >
2.0
<506
0.8
0.6-
0.6
0.6
0.6
Normal Correlizeedeeeee
0.0
0.0
0.0:
150
200
300
250
30
350
Structura Takon ID
Structure
Structure Token
Figure 7: Example of performance variation across the selection of different structural tokens in Tox21 dataset.
26

---

Published as a conference paper at ICLR 2026
BACE Cass: posttive case 390
BВBP no: positive case * 806
ВВВP. np: positive case 529
Normalzed Wkathood oftaxen True"
Tow
<506 -
1.0
<506 P
1.0
<50
Normalzed Ikalhood oftouen
0.0
Nomalize
0.2
0.2
0.2
0.0
0.0
100
200
100
200
160
200
Structure Token ID
Structure Token ID
Structure Tokon ID
BBBP pprpp posilive case 908
CinTox FDA APPROVED:
ClinTrr FDA_AAPRRVEDDDD.D posilive case 138
Nommalzze loee of TTuee
Ikellioel of tokan "Tuee"
1.0
<50
<50gm>>
1.0
<50G
Nomeltad Ioelood token
0.8
0.8
0.6-
0.6
Normafized (kellpoo
0.4
0.4
02
50
100
150
200
250
1.00
150
200
250
250
Token
Structure
ID
Structure Token
Figure 8: Example of performance variation across the selection of different structural tokens in BBBP, BACE and ClinTox dataset.
HIV HIV a__.actiee postive case 103
HIV HIV_HlV_adivv posilive
910
Tox21_N-AA: posilvv 157
Normalized Ihelheod of token True
1.0
<50
<506m1>>
1.0
<505111
Noma ltwethood token
0.8
0.8
0.8
Normalzed
50
100
200
100
200
250
150
250
Structure
Tox21. NRsAR: positive
NR-AR-LBD positive
Tox21_ NR-Aromatase: positve case N
Noraaaadd Ikelhood oftokan Trual
Normaizee Ikelhood tokknn TTrae"
1.0-
<50511;>
1.0
450Gmm
2.0
4506mm
Nommalzed Ikelhood token
0.8
0.8
0.8
0.6-
0.6
0.6
0.4
50
100
150
200
250
100
150
200
250
50
100
150
200
250
Token
Figure 9: Example of performance variation across the selection of different structural tokens in Tox21 and HIV dataset.
27

---

Published as a conference paper at ICLR 2026
Scaffold
Molecule #1
Molecule #2
Molecule #3
Structraa Token: <50G
BBBP: 978
B88P. N979
BBBP: 980
CICECNICCCCN ICCNicoeaaa)CC
Structural Token: <50G??
Tox21: 5328
Tox21: # 5329
Tox21: A 5390
Structural Token: <505mz?
Tox21: e 5498
Tox21: # 5497
Tox21: e 5496
c.cocclslessl
Structural Token: <
HIV. it 19657
HIV: # 19656
HIV: # 19859
Structural Toker: <sog>?
HIV: 20245
O' HIV: 20246
HIV: 20247
Structural Taken: <50G
Structural Token: x50
20893
HIV: 20694
Figure 10: Molecules in same training set sharing the same scaffold with same structural token.
28

---

Published as a conference paper at ICLR 2026
Scaffold
Molecule #1
Molecule #2
Molecule #3
Structural Token;
Tox21: 5771
Tox21: # 5772
8BBP: 1083
CICCCOCCCHII CHRAIIII
Structural Token; <s0G
BBBP, 1100
BBBP: 1101
ClinTxx: 1031
Structural Token; <50Gs1>
Tox21: e 5348
Tox21: 5347
BBBP: 1542
Tosooaannnnnnooccccc Ccbeolatee
Structural Token: <50Gmm? ?
Tox21: # 9551
ClinTox: # 864
ClinTox: # 866
N1
HU
claseallCcaaajcccooo
Structural Takan: <S0G
Tox21: 5855
Tox21: 5856
HIV: 30868
QNeleead Ho-Y--yoojeeeeeeee
<500
HIV: # 23379
HIV: # 23380
Tox21:
eleeef-chomt
Figure 11: Molecules in different training sets sharing the same scaffold with same structural token.
29


Reviews:
[{"review_id": "uN5IX6F9aF", "reviewer": "Reviewer_kfpk", "summary": "As LLMs are facing challenges with graphs because of their structural hallucination. Existing approaches have their apparent shortages. This paper proposes a new method that insert a special token called  <SOGk > to represent the structure of graph better. In this paper, the author proposes a special tokenizer to generate a highly selective token based on graph topology, then align the new <SOGk > token with the existing tokens by a set of hybrid structure Question-Answering corpora. The experiments results show it significantly helps LLMs to understand graph datasets better.", "questions": "NA", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper innovatively creates a new method in the graph recognition of large language models (LLMs). Unlike previous methods, which usually convert the entire graph data into another format, the Structure - Oriented Graph (SOG) provides a novel way for LLMs to understand textual graphs through a special token that reflects the overall structure of the graph.\n\nThe advantage of SOG is that it uses a Graph Neural Network (GNN) as an encoder to obtain deeper information within the graph data. To address the drawbacks in converting continuous embeddings, SOG maps the continuous representations into a K - discrete token vocabulary, which is helpful in solving misalignment. It seems to be quite original.", "weaknesses": "Method proof missing: This paper only elaborates the main idea of SOG but lacks some basic proof regarding how this method emerged. From an objective perspective, some basic algorithms need to be listed to prove how the components of SOG evidently improve the accuracy and efficiency in graph learning.\n\nThe Hybrid QA method is not always the best way to align a new single token with existing tokens. The chart in this paper shows that in some cases (BBBP, ClinTox), Hybrid QAs are slightly worse than KNN matching alone. This indicates that SOG still needs to find a better way to align its structural token with other tokens on more complex datasets, especially when fine - tuning with description - token.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZDTTIOdSs3", "reviewer": "Reviewer_qiqx", "summary": "The authors propose to use a special token <SOG_k> to represent the entire topology of a graph, so that LLM can use that token to make predictions. The method builds a structural tokenizer using GNNs and quantizes the representation into one entry. In addition, the authors construct hybrid structure QA corpora to align the new structural tokens with text tokens using LoRA. On five MoleculeNet-style graph-level classification datasets, their method improves over zero-shot/few-shot/LoRA/soft-prompt LLaMA baselines by a large margin.", "questions": "1. What is the exact size, structural diversity, or negative-sampling strategy of those three QA corpora?\n2. Is there any out-of-vocabulary structure case?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The problem is clearly defined with proper related works.\n2. Using one token for each graph is a convincing idea.\n3. The overall tokenizer pipeline is clear and meaningful.\n3. The authors show strong empirical gains on 3B and 7B models.", "weaknesses": "1. All main experiments are on small to moderate molecular graphs, where graph sizes and motif variety are relatively small. It’s unclear how this scales to large real-world graphs.\n2. Compressing a large graph into a single token may lose information.\n3. I did not see the training cost analysis.\n4. Hyperparameter sweeping for the baseline is not reported.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "acE5clunMG", "reviewer": "Reviewer_tD9C", "summary": "The paper introduces a method that maps each graph’s topology to a single discrete structural token <SOG_k> that can be directly fed into an LLM, avoiding long graph-to-text encodings.\nIt builds these tokens through a topology-aware tokenizer and aligns them with the LLM’s embedding space using structure-based QA fine-tuning.", "questions": "- Generalization: How does SOGk perform on non-molecular benchmarks (e.g., social/citation/synthetic graphs) and on node- or link- level tasks?\n\n- Invariance: Which parts of the tokenizer are provably permutation-invariant? What breaks if node ordering or anchor selection is adversarially perturbed?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- One discrete token injects topology into the LLM, drastically reducing prompt length vs graph-to-text.\n\n- Large improvements on multiple MoleculeNet tasks with clear ablations.\n\n- Interpretability analysis is insightful.\n\n- Works with off-the-shelf LLMs (LoRA + token embeddings), no architectural changes to the base model.", "weaknesses": "- It remains unclear whether this method scales robustly to heterogeneous or large real-world graphs (e.g., knowledge graphs, citation networks, or synthetic structural datasets).\n\n- The approach is not novel. Some papers [1,2,3,4] have already provided similar approaches. Moreover these baselines are missing.\n\n- Permutation/anchor bias: Tokenizer choices (anchor selection, hop labeling) may introduce ordering biases; robustness to isomorphisms not rigorously tested.\n\n- Not generalizable. Requires fine tuning for each new dataset.\n\n[1] Perozzi, Bryan, et al. \"Let your graph do the talking: Encoding structured data for llms.\" arXiv preprint arXiv:2402.05862 (2024).\n\n[2] Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and\ngraph assistant. arXiv preprint arXiv:2402.08170, 2024b.\n\n[3] Chai, Ziwei, et al. \"Graphllm: Boosting graph reasoning ability of large language model.\" arXiv preprint arXiv:2310.05845 (2023).\n\n[4] Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson,\nand Bryan Hooi. G-retriever: Retrieval-augmented generation for textual graph understanding and\nquestion answering. arXiv preprint arXiv:2402.07630, 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IKq08HXoV5", "reviewer": "Reviewer_g287", "summary": "To address the structural hallucination problem of LLMs)on graph-structured data—where existing Graph-to-Text approaches suffer from high token consumption and attention dilution, and Graph-to-Embedding methods face cross-modal misalignment—this work proposes a novel solution: introducing a **single special token** `<SOGₖ>` . A topology-aware graph tokenizer maps the entire graph topology into a highly selective discrete token from a learnable structural vocabulary of size K, which is then injected into the LLM input sequence, forming a hybrid architecture that blends structural and textual modalities.", "questions": "1. **Multi-token extension**: The single-token design is indeed attractive. However, has the authors explored using **multiple structural tokens** (e.g., 2–5 tokens) to capture richer or hierarchical graph properties? Could this further improve performance on complex graphs?\n\n2. **Relation to prior work**: How does this method fundamentally differ from *LangTopo* and *Dr.E*? Is the key novelty the **end-to-end joint training with LLMs in a shared token space**, or the **specific topology-aware tokenizer design**? Clarifying this distinction would strengthen the paper’s positioning.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- **Innovative and elegant design**: Representing an entire graph via a single virtual global node mapped to a vocabulary index is conceptually clean and computationally efficient.\n- **Unified representation space**: By training on hybrid QA corpora (including k-NN matching, true/false judgment, and description alignment tasks), `<SOGₖ>` is embedded in the **same semantic space** as native LLM text tokens, effectively eliminating the modality gap inherent in Graph-to-Embedding methods.\n- **High efficiency and strong performance**: The approach achieves competitive or superior results with minimal token overhead (just +1 token) and negligible parameter cost.\n- **Interpretability**: The paper includes meaningful analysis showing that each `<SOGₖ>` token occupies a distinct region in the embedding space, suggesting meaningful structural encoding.", "weaknesses": "- **Insufficient related work discussion**: To the best of my knowledge, similar ideas have appeared in prior literature(1) *Multi-View Empowered Structural Graph Wordification for Language Models*, which also uses discrete tokens to represent global graph structure; (2) *LangTopo*\n  The paper does not adequately differentiate itself from these works.\n- **Lack of ablation studies**: The proposed “topology-aware graph tokenizer” comprises three key components—(i) anchor node selection (via node degree), (ii) virtual global node , and (iii) self-supervised topological reconstruction—but none are ablated independently to assess their individual contributions.\n- **Missing hyperparameter sensitivity analysis**: The structural vocabulary size K is fixed at 256 without exploring alternatives (e.g., K=64, 128, 512). It remains unclear whether performance saturates, degrades, or improves with larger/smaller K, raising concerns about robustness and scalability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)