Entropy-Guided Dynamic Tokens for Graph-LLM Alignment in Molecular Understanding
Authors: Zihao Jing
QIUHAO Zeng
Ruiyi Fang
Yan Sun
Boyu Wang
Pingzhao Hu
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Modeling
Graph–LLM Alignment
Molecule Understanding
Backbone-Free Tuning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: EDT-Former: entropy-guided dynamic query tokens map molecular graphs to LLMs, capturing local and global structure features for comprehensive understanding and reasoning with backbone-free, connector-only training.

Abstract: Molecular understanding is central to advancing areas such as scientific and drug discovery, yet Large Language Models (LLMs) struggle to understand molecular graphs effectively. Existing graph–LLM bridges often adapt the Q-Former-style connector with fixed-length static tokens, which is originally designed for vision tasks. These designs overlook stereochemistry and substructural context and typically require costly LLM-backbone fine-tuning, limiting efficiency and generalization.
We introduce **EDT-Former**, an **E**ntropy-guided **D**ynamic **T**oken Trans**former** that generates tokens aligned with informative molecular patches, thereby preserving both local and global structural features for molecular graph understanding. Beyond prior approaches, EDT-Former enables alignment between frozen graph encoders and LLMs without tuning the LLM backbone (excluding the embedding layer), resulting in computationally efficient finetuning, and achieves state-of-the-art results on MoleculeQA, Mol-Instructions, and property prediction benchmarks (TDC, MoleculeNet), underscoring its effectiveness for scalable and generalizable multimodal molecular understanding.

Published as a conference paper at ICLR 2026
ENTROPY-GUIDED DYNAMIC TOKENS FOR GRAPH- LLM ALIGNMENT IN MOLECULAR UNDERSTANDING
Zihao Jing', Qiuhao Zeng', Ruiyi Fang', Yan Sun', Boyu Wang', Pingzhao Hu!
DDepartment of Computer Science. Western University, London, ON, Canada
2 Department of Biochemistry, Western University, London, ON, Canada j ing2 wwingg9ewwo.com
Swwwo.a........ phu49euwo.ca
ABSTRACT
Molecular understanding central to advancing areas such as scientific discov- ery, yet Large Language Models (LLMs) struggle to understand molecular graphs effectively. Existing graph-LLLM bridges often adapt the Q-Former-style connec- tor with fixed-length static tokens, which is originally designed for vision tasks. These designs overlook stereochemistry and substructural context and typically require costly LLM-backbone fine-tuning, limiting efficiency and generalization. We introduce EDT--Former, an Entropy-guidee Dynamic Token Transformer that generates tokens aligned with informative molecular patches, thereby preserv- ing both local and global structural features for molecular graph understanding. Beyond prior approaches, EDT-Former enables alignment between frozen graph encoders and LLMs without tuning the LLM backbone (excluding the embed- ding layer), resulting computationally efficient finetuning, and achieves state- of-the art results on MoleculeQA, calecule-oriente Mol-Instructions, and prop- erty prediction benchmarks (TDC, MoleculeNet), underscoring its effectiveness for scalable and generalizable multimodal molecular understanding.
Former
Mol-Llama
Mol-Instruct.
3D-MoLM
Frozen Backbone
Finetune Backbone
1
4.8-5x10" FLOPPSTTken
90
EDT-Former-8.3B
82,3
GPT-5
80
Galactica-6.71
74.5
Mol-Instructions-BB
72.5
72,9
Mol-Llama-3.1-88
70
3D-MoLM-8B
Accuracy (%)
60
$1.2
50.7
50
48.6
462
43.1
40
30
20
Pampa
BBBP
Structure
Source
Property
Application
Figure 1: LLM joint fine-tuning efficiency and benchmark performance preview of EDT-Former.
"Corresponding author. Departments of Computer Science and Binchemistry, Western University, 1400
Western Road, London, Ontario N6G 2V4, Canada. E-mail: phu49@ (P.H.)

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Background. Large language models (LLMs) such as Llama (Touvron et al. 2023a MetaAl 2024) and GPT (OpenAI 2023 2025) series demonstrate powerful multimodal reasoning across domains. This rapid progress highlights the potential adaptation to specialized scientific domains, and NatureLM (Xia et al. 2025) exemplifies this direction in scientific knowledge. Within molecular science, recent foundation models have extended large-scale pretraining to molecular graphs and geometry structures. Uni-Mol-v2 (Ji et al., 2024) leverages extensive conformer datasets to jointly model atomic, graph, and geometric representations, while UniMolT (Zhang et al. 2024) adapts instruction-tun LLMs to molecular reasoning and understanding tasks.
To bridge molecular graphs with natural language, multimodal connectors have been introduced. Most approaches adopt the Q-Former mechanism (Li et al. 2023), where a fixed number of monality-anchor tokens are learned query structural encoders. Representative examples include 3D-MolTS (Pei et al. (024b)) Mol-LLM (Lee et al., 2025), and Mol-LLaMA (Kim et al.) 2025), which demonstrate the feasibility of aligning molecular structures (graph and geometry) with lan- guage models and enabling cross moosl molecular understanding.
(a) Small Mol: N=16
(b) Larger Mol: N-50
Encoder
Q-Former
Fix length tokens
Encoder
O-Forer
Fix length tokens
Len =
Len
@: Could you please count the number of carbuxyl groups (- COOHI) are the molecule <mol>?
Q: Could you please count the number of carboryy groups (-COOH) that are in the molecule <mol>>
LLaMA3 LLaMA3
G: carboxyl groups are in this molecule.
carboxyl groups are this molecule.
A: In total, it has 2 -COOH
A:...I0 COOH are in the molecule.
@: In <mol>, which backhone carbons are the carboxyl groups attached?
@: Given the molecule <mol> To which nitrogens the five acetale attached?
COOl on lerminal
G: NI, N2, and N3 (total 5)
COOH CL/C4
A: ...as a result, cannot determine...
Figure 2: Illustration of motivation of structure. Comparison of molecules of different sizes (atom counts N 16 and N 50) encoded by the same fixed query length Q-Former bridge (8 query tokens) to I.lama3.1 1.mmna-3.1-85 1-8B backhone, with example prompts and generated responses.
Challenges. (1) Loss of structure. Current graph-LLM bridges typically fuse modalities by in- troducing fixed number of learnable query tokens, which interact with molecular encoder outputs and are then fed into the language model. (Lee et al. al. 2025 Kim et al.) 2025). However, compressing seagghh-hterogeeeuus molecules into the fixed set collapses critical features such as stereochemistry and functional groups. This loss carries over to reasoning, where even
prompting provides little gain and can reduce accuracy on property prediction (Kim al. 2025). As shown in Fig. 2, fixed-length fusion captures key groups in small molecules (N=16 atoms) and supports structure-aware tasks; however, in larger molecules (N=50 atoms), results in incomplete substruc- ture coverage, information loss, and consequently brittle, chemically unfaithful predictions.
(2) Heavy fine-tuning. Most prior systems train the bridge jointly with the LLM, requiring gradient updates the backbone itself. Such updates hinder generalization: models overfit to narrow datasets, lose robustness across modali- ties and structural variations, and fail to transfer alignment when scaled to larger frozen backbones. Clearly, they are also computationally inefficient: shown in Tab. [ jointly tuning the LLM and connector requires x96 more trainable parameters than tuning the connector alone (details are in App. D.3). Consequently, a connector-onle bridge with frozen encoders and frozen LLM is urgently needed achieve scalable, low-cost deployment.
Table 1: Computing costs comparison between frozen/unfrozen backbone settings of Mol-I.lma..
Connector LLM Trainable FLOPs/Token Time/Step
Finetune Finetune
8.IB
4.9e10
0.93
Finetune Frozen
84M
1.7e10
2

---

Published as a conference paper at ICLR 2026
Our approach. We propose EDT-Former, Entropy serropy-guided Dynamic Token Transformer that aligns chemical graphs with frozen LLMs while preserving structural fidelity. EDT-Former intro- duces an Entropy-Guided Patching strategy that segments molecules into informative sub- groups, generating dynamic query tokens that retain stereochemistry. Query Transformer module that integrates these dynamic tokens with learned modality anchors to form a stable cross-modal interface before mapping into the LLM embedding space. Together, these designs enable Trozen-backbort alignment, yielding efficient training, robust generalization, and chemically faithful understanding (Fig. Our contributions are summarized as follows:
further incorporates the Dynamic
This work presents EDT-Former, the first connector-only method that aligns chemical graphs with frozen LLMs via dynamic, substructure-awann query tokens.
We present Entropy-Guided Patching and Dynamic Query Transformer, which together en- able efficient cross-modal alignment without updating backbone parameters.
EDT-Former achieves state-of-ther of--hh--rtttttttoo results on molecular understanding and property prediction benchmarks, demonstrating both scalability and robust generalization.
RELATED WORK
Multimodal Fusion for Molecular Generative Modeling. Beyond SMILES- models (e.g.,
KV-PLM Zeng et al. 2022) and Mol-Instructions (Fang et )023)), recent work has attempted to fuse SMILES, topological graphs, and 3D conformers with large language models. GIMLET ZZaao 2023) showed that instruction tuning over paired graph-text corpora can provide LLMs with basic molecular understanding. Larger frameworks extended modality coverage: HIGHT (Chn
ct al.
et al.
2024) uses hierarchical BRICS- based graph tokenization approach, and MoleculeSTM (Liu
ct al.
2022). MolFM (Luo et al. 2023a BioTS+ (Pei at. 2024a), and ProI.LaMA eL. 2024)
combine discrete structure tokens, biomedical knowledge graphs, and protein sequences, while 3D> MolT5 (Pei et al.) 2024b) incorporates coarse 3D geometry for conditional generation. Most re- cently, UniMolT Zhang et al. 2024), Mol-LLM (Lee al. 2025). and Mol-Llama (Kim et al.) 2025) adopt the Q-Former mechanism from BLIP-2 (Li et al. (023) to bridge molecular graphs and LLMs, relying on fixed-length learnable modality anchor tokens. Such fixed-length fusion compresses stereochemistry and graph substructures, leading to structural information loss. More flexible alignment strategies needed to retain sub-structural fidelity as molecular complexity and conformer diversity increase.
Molecular Understanding with Multimodal LLMs. Molecular understanding has been explored by combining structural encoders with instruction-tuned LLMs. MolReasoner Zhao et al. (2025), StructCoT (Jang al.) 2024) showed that coupling molecular tokens with prompts enables step-wise reaction explanation and property inference. Subsequent efforts introduced structural bias, such as Mol-LLM (Lee al.] 2025) with topology-conditions attention and ProLLaMA (Lv et al. 2024)
with protein-ligand co-representations. However, most approaches still depend on fixed-length an- chor token representations or extensive fine-tuning of the LLM, which weakens structural fidelity and makes training inefficient. Recent chemical agents, such ChemCrow (Bran al.) 2023) attempt to add planner feedback but remain constrained by conformer sensitivity and limited scala- bility. Progress, therefore, requires multimodal alignment method that preserves modality-specinee cues without relying on heavy LLM tuning and adaptation. More related works and relationships to current works are discussed in App.
3 EDT-FORMER: ENTROPY-GUIDED DYNAMIC GRAPH-LLM ALIGNMENT
CDTT-Former provides efficient and substructure-awant alignment between molecule and LLM. Sec. 3.1 outlines the overall architecture, after which Sec. 3.2 introduces the Entropy-Guides Patch- ing strategy for dynamic token generation from graph node-embeddings, while Sec. 3.3 presents the Dynamic Query Transformer module. Finally, Sec. deseribes the training process under the bozennbackbone regime. The implementation details are discussed in App.
3

---

Published as a conference paper at ICLR 2026
Entropy-Basse Based Patching
EDT-Former
Dynamic Query Transformer
Node Embeddings
3D Geo.
Flod Graph Exceder
Nout Level Embeddiing
Topological
ITN
FFN
Share
Describe the Emolecules substractures...
Crees Atm
Crose Alm
103.......
Self-Attn
Self-Attn
Sharc
Large Language Model
Sylitinn
Patches
Dymamic
molecule containa group, carbasylic acid group. an aromatic benzene ring..
Dyramic
Paoling
Entropy-
Length
Pooj
Nett-Anam Predictor
Paicchin
Figure 3: The architecture of EDT--Former (a) Entropy-based Patching segments node embeddings into patches produce dynamic query tokens. (b) EDT--Formr integrates anchors and dynamic queries through Dynamic Query Transformer to align the molecular graph with the LLM.
Entropy
2.0
1.4
1.2
OCOCS & Č Č CCNCCCCCCCCOO
Atees
Figure 4: Illustration of entropy-guidee guided patching on an example molecule. Atom-level entropy is plotted along the molecular sequence, and patch is initiated after each local maximum.
3.1 STRUCTURE OVERVIEW
EDT-Former establishes alignment between molecular encoders and frozen LLMs through com- plementary components: Entropy-Guided Patching and Dynamic Query Transformer. As shown in Fig. node embeddings from frozen graph encoders are processed by entropy-guided patching, which measures node-level uncertainty for language models, segments molecules into informative sub- grougg and pools them into dynamic tokens that preserve local graph features. These tokens are combined with static modality anchor tokens in a shared query bank, where self-attention propagates context and cross-attention retrieves structural evidence from molecular embeddings. The resulting representations are projected into the LLM embedding space, forming cross-modal alignment in- terface that balances global consistency from modality anchors with local fidelity from dynamic query tokens, enabling modality alignment with high efficiency and structural fidelity.
3.2 ENTROPY-GUIDED SUB-GRAPH PATCHING
Setup. As shown in Fig. D a), let molecule be represented by a SMILEES-ordered atom sequence (a,...., a.). We pre-train lightweight Next-Atom Predictor fo (NAP, small Transformer) on large canonical SMILES corpora to model p(a,+1 (1:z). At inference time, given logits L; R over the atom vocabulary V. the probability assigned to the ground-truth next atom is
Pt softmax((L))
t= 1,
The information content is defined (negative loglliiklihodd as
e, log (nats; use log, for bits desired).
(2)
Peak-based segmentation. Instead of thresholding, we identify local maxima of the entropy sig- nal } and place split points at peaks (shown Fig.
peak(() = (e-1 <() A (e.> (c)>441))
optionally enforcing minimal separation A (non-maximum suppression) and a small prominence y remove spurious bumps:
NMS()) = min mss e, 2 es
prom(!) = (er +e+1) 2 7]. (4)
4

---

Published as a conference paper at ICLR 2026
Algorithm 1: Dynamic Query Transformer (anchors + dynamic tokens)
Input : -Grahh G; frozen graph encoder Σ; frozen LLM L; dynamic tokens Z [:1,.... , AM] from Sec. 3.2 #anchors k; layers L: projector pmnj-
Output: U E IR (x++Jxxxdux (LLM-conditioning sequence).
Node embeddings: X + E
// frozen molecular encoder
Anchors: Qix E -R
1/ learnable, modality-stable
Query bank: Qbank < 1Qix Z] E (k++)))
// construct query bank
Multimodal Alignment (L layers):
11 Stack of Transformer Blocks
for E f- to L do
// mix anchors dynami Qbank t Qbenk + Cross-Attn( Qbanks K = = V-X); // retrieve node-level evidence
Qbank + Qbank + Self-Attn (hank):
Dank + Qbank + FFN((bank))
11 shared FFN for all query tokens
Projection to LLM space: U < Wproj Qbank: return U:
// (++M)
// LLM consumes U; & frozen; train bridge only
Then, the valid split indices are collected T, E (2...., - 2) peak(() NMS(L) A prom(t) } and form dynamic segments by cutting after each peak:
1 To 71
TM = { JM-1) T., S₁ {t) T--1 nh). (5)
Graph mapping and pooling. Let (I....,7) - - (1, (I...., ......,N map SMILES positions to graph node indices. Each segment S₃ induces a substructure (node set)
= ((1) L S& E [
Given frozen graph-encoder node embeddings X E Rdx, we construct one query token for each substructure via average pooling:
AvgPool([ i Sz)) E Rd, k I= M.M
Zk
The collection { M 1= constitutes dynamic query tokens for Dynamic Query Transformer.
Rationale. The entropy e quantifies the predictive uncertainty of extending the current SMILES atom with de+-1 under lightweight Next-Atom Predictor (NAP). Local peaks indicate positions that are hard for a SMILES-based sequence model to predict, and grouping tokens between peaks yields data-driven patches that mark information-dense segments in the 1D sequence, rather than exact chemical subgraphs. Since SMILES is generated via a DFS traversal of the molecular graph, ad- jacent tokens typically reflect nearby atoms, so these entropy patterns still inherit meaningful local structure while remaining strictly sequence-based and aligned with the LLM's view. This entropy- driven segmentation is simple, reproducible, and automatically matches dynamic-token count to molecular complexity. The theoretical analysis of the rationale of Entropy-Guided Patching is dis- cussed in App. A.I and the detailed analysis is in App. D.9
3.3 DYNAMIC QUERY TRANSFORMER
Dynamic Query Transformer integrates fixed modality anchors with dynamic substructure tokens to form compact, structure-awe aware interface to frozen LLM,, as shown in Fig. 3)). Let Nxd be node embeddings from the frozen graph encoder and let Z = [æ1,.... 2Md E R Mxd be dynamic tokens from Sec, 3.2 As shown in Algorithm we initialize & learnable anchors Qix E & and build query bank Qbant [Qixi Z] (K+++ Myxd A lightweight transformer with layers refines Qbank by (i) self-attention over queries to mix global and local context, (ii) cross-attention from queries to node embeddings X to retrieve substructure evidence, and (iii) a shared ferw-ror network. The enriched queries are projected to the LLM embedding space via U Wpmnj Qbank E and then consumed by the frozen LLM conditioning. During training, only the bridge parameters (anchors, attention/FFN, and W Wpwsj) are updated; both the molecular encoder and the L.L remain frozen, yielding efficient gLM alignment.
5

---

Published as a conference paper at ICLR 2026
Table 2: Zero-shot accuracy (%) on molecular property prediction tasks from MoleculeNet and TDC benchmarks, where each value is averaged over 13 shared prompts per task for fairness. Prompt settings details are in App. D.5 The best (pink) and second-bes (lightpink) results are highlighted.
Model
Size
PAMPA
BBBP
BACE
CLINTOX
DILI
HERG
HIA
HIV
AMES
PGP
General LLMs
GPT-4o
50.52
63.52
39.48
26.34
53.47
64.04
76.00
17.02
53.59
50.21
Llama2
69.27
49.06
38.96
26.05
51.26
65.02
76.66
15,20
53.08
51.09
Llama3.1
58.00
55.64
40.44
23.93
52.31
64.28
78.07
16.34
53.46
50.99
Molecudar L.Ms
3D-MoLM
54.50
$1.20
41.43
25.92
48.78
62.67
72.93
26.16
53.01
51.55
LLaMo
53.09
57.18
41.91
25.55
49.08
62.77
72.75
24.39
$1.67
51.22
Mol-Ins.-Llama2
40.78
51.22
42.61
25.99
48.91
62.79
72.78
22.82
53.72
Mol-Ins--Llamad.
57.39
53.29
40.12
26.79
52.39
47.23
50.95
16.70
48.98
Mol-LLaMA-2
7.2B
74.23
53,99
41.54
27.96
51.92
65.60
87.88
22.51
52.97
51.14
Mol-LIaMAA3.1
8.3B
67.15
56.64
38.68
23.07
51.76
70.61
78.25
50.73
Ours
EDT-Former
8.3B
82.34
72.48
49.12
56.55
50.27
73.46
82.15
46.56
55.20
54.64
3.4
ALIGNMENT TRAINING
EDT--Former is trained via a two-stage regimen (pretraining, then alignment tuning) on the Mol- LLaMA- dataset (Kim et al. 2025). In pretraining, only the Dynamic Query Transformer is optimized together with frozen graph encoders, without attaching the LLM. Objectives follow standard adapter-style practice (ef. Q-Former): cross-modal contrastive loss for global cohesion, an anchor-modality matching loss to stabilize the fixed interface, and masked substructure recon- struction loss inject local chemical semantics. Details discussed in App. C.2
In the alignment tuning stage, the frozen LLM is attached while the graph encoders and LLM re- main frozen; only the parameters of Dynamic Query Transformer updated to align structure- aware queries with instruction prompts. This protocol preserves backbone stability, enables efficient adaptation, and yields a chemically faithful graph-LLM interface. Detail are in App. C.3
EXPERIMENTS
In this section, EDT-Former is evaluated on benchmarks MoleculeQA (Lu et al. 2024), Molecule- oriented Mol-Instructions (Fang al. 2023), Pampa, and BBBP from TDC Huang al. (2021)
addressing the following questions: Q1) Does EDT-Former outperform existing multimodal molec- ular LLM baselines? Q2) Can Entropy- Guided Patching capture subgraph features? Q3) Does the Dynamic Query Transformer improve LI.M understanding compared to fixed-length query connec- tors? Q4) Does freezing the LLM reduce compute while retaining strong accuracy? See App. D for baseline and dataset descriptions, comprehensive experiment implementation, benchmark and evaluation details.
4.1 PROPERTY PREDICTION TASKS
As shown in Tab. 2 evaluate ten molecular property prediction tasks from TDC (Huang al., 2021) and MoleculeNet (Wu et al. 2018) benchmarks using official splits and zero-shot inference. Comparisons cover general LLMs and molecular LLMs (including multimodal variants). To reduce prompt sensitivity. the average accuracy over 13 prompt settings is reported, with all the prompt settings per-prompt accuracies in App. D.5). Under identical conditions, EDT-Former attains the best results on both tasks, with 200 relative gains over the strongest baseline and pushing average accuracy above 70% on BBBP, HIA, and PAMPA datasets, indicating practical viability for LLM- based property prediction (Q1). Potential dataset imbalance is discussed in App. D.11
4.2 MOLECULEQA (REASONING AND UNDERSTANDINC)
Evaluation conducted on the large-scale MoleculeQA benchmark (Lu et al., 2024) (Tab. 3). multiple-choice suite covering four tasks- source, property, and application. Large GPT

---

Published as a conference paper at ICLR 2026
Table 3: Performance on the MoleculeQA benchmark. Models are compared across four tasks (Structure, Source, Property, Application) with accuracy (%) reported. The best (pink) and second- best (lightpink) results are highlighted. Modality T/G/3D represents text, graph, and 3D geometry.
Model
Modality
Imp.
Size
Stret.
Prop.
App.
Avg.
Total
Random
24.41
22.30
23.04
24.57
23.58
24.03
Molecular LLMs
BieMedGPPT-M
Text
SFT
7B
54.19
60.01
38.85
40.90
48.49
52.23
Mol- Inst.-nnaaaannt Llama3.1
TAG
SFT
72.79
70.82
43.08
41.22
56.98
65.31
3D-MoLM
T/G/DD
SFT
73.17
70.50
44.79
44.19
58.16
65.96
LLaMo
Text
SFT
70.56
66.63
44.60
45.18
56.74
63.74
Mol-Llama3.1
TexuGrrahh/D
SFT
8.2B
73.16
70.22
45.70
46.18
58.82
66.21
General LLMs
Galactica
Text
SFT
6,7B
32,35
41.92
31.05
28.21
33.38
33,96
BLOOM
Text
SFT
7.1B
35.01
47.51
31.46
33.56
36.89
37.31
Pythia
Text
SFT
42.79
38.58
39.07
44.84
Lluma-2-chat
Text
SFT
28.75
39.84
31.33
27.71
31.91
31.54
Vicuna-v1.5
Text
SFT
13B
37.01
43.19
30.64
31.55
35.60
37.07
Large Scale General LLMs
GPT-3.5
Text
10-Shot 10-Shot
25.60
37.60
28.04
32.22
30.87
29.29
GPT4
60.94
50.19
35.57
43.91
47.65
GPT-5
Text
62.78
53.22
36.42
46.91
Ours
EDT-Former
T/G/3D
10-Shot
8.3B
66.46
60.98
40.35
36.40
51.05
58.78
EDT-Former
T/G/3D
SFT
8.3B
74.55
72.39
50.71
48.58
61.56
68.34
Table Results on Mol-Instructions dataset. Models are finetuned and evaluated on two tasks: molecular description generation (BLEU, ROUGE, and METEOR scores) and molecular property prediction (MAE). The best (pink) and second-best (lightpink) results are highlighted.
Molecular Description
Models
Property (MAB)
BLUE 2
BLUE
ROUGE:
ROUGB-2
ROUGE
METEOR
Alpaca-77 7B
0.068
0.014
0.178
0.041
0.136
0.107
322.109
Baize-7B
0.064
0.015
0.189
0.053
0.148
0.106
261.343
LLaMA-2-78
0.059
0.014
0.164
0.066
0.148
0.184
5.553
Vicuna-v1.55 1-513B
0.052
0.011
0.151
0.055
0.130
0.168
860.051
Galatica-6.7B
0.063
0.065
0.568
Qwen3-8B
0.098
0.029
0.207
0.050
0.157
0.173
4.737
Mol-Inst-Llama2-7B
0.217
0.143
0.337
0.196
0.291
0.254
0.013
Mol-Inst.-
0.646
15.059
Mol-LLaMAZ-7721
0.433
0.385
0.711
0.649
0.601
0.601
0.0087
Mol-LLaMA331-4 1-8.8
0.445
0.398
0.717
0.656
0.709
0.617
0.0079
MolReasoner-7B
0.438
0.322
0.553
0.366
0.482
0.475
10.323
EDTT-Former-8.38
0.424
0.402
0.726
0.652
0.717
0.631
0.0062
models assessed in 10-shot setting, while all other models are fine-tuned the official splits for the same number of epochs under matched hyperparameters. Under these conditions, our SFT model consistently achieves the best performance across all four tasks, demonstrating strong molec- ular reasoning. Notably, the 10-shot variant EDT-Former outperforms the newest GPT-5 model, indicating efficient trade-off between scale and domain alignment. These results answer Q1.
4.3 MOL-INSTRUCTIONS (MOLECULE-ORIENTED)
For benchmark Mol- Iottrunstucsonss (Fang et al. 2023), we evaluate all 6 molecule-oriented and open- question tasks. For fair comparison, all baselines and EDTT-Former are fine-tuned under identical settings on the official splits. Tab. tion, Property prediction, Retrosynthesis, Forward reaction prediction, and Reagent prediction tasks (with Description-guides molecule design task reported in App. D...................... EDD-Former attains stronger chemical ability on most metrics of all the tasks, indicating improved performance relative to both general and molecular LLM baselines. The detailed experiment settings and metrics are discussed in App D.3 We further investigate the influence on natural language ability of EDT-Former and Mol-I (Kim al. 2025) when performing fine-tuning in D.p These address Q1.
show the performance on Molecule description genera-
D.7

---

Published as a conference paper at ICLR 2026
Table 5: Results the Retrosynthesis task from the Mol-Instructions benchmark. Models are finetuned and evaluated on the official splits, reporting metrics as defined by the benchmark. The best (pink) and second-best (light pink) results are highlighted.
Model
Exactt
BLEUT
Levenshteing
RDK FTSt
MACC FTS*
Morgan FTSt
Validity
Alpaca
0.000
0.063
46.915
0.005
0.023
0.007
0.160
Baize
0.000
0.095
44.714
0.025
0.050
0.023
0.112
ChalGLM
0.000
0.117
48. 365
0.056
0.075
0.043
0.046
LLaMa
0.000
0.036
0.018
0.029
0.017
0.010
Vicuna
0.057
46.877
0.025
0.030
0.017
Galactica
0.000
0.452
34.940
0.167
0.274
0.134
0,984
Text+Chem
24.043
0.765
0.585
0.698
Mol-Ins-LLamma2
31.227
0.487
0.230
Mol-Ins--Llmma..
0.333
0.842
17.642
0.704
0.815
0.646
1.000
Mol-I.Lam--3.1
0.340
0.877
38.324
0.708
0.822
0.601
1.000
HIGHT
0.008
0.863
28.912
0.564
0.340
0.309
1.000
EDT-Fommer 0.387
0.930
34.202
0.721
0.836
0.670
1.000
Table Results on the Forward Reaction Prediction task from Mol-Instrustions benchmark. Models are fine-tuned and evaluated on the official splits, reporting metrics as defined by the benchmark. The best (pink) and second-best (light pink) results are highlighted.
Model
Exact
BLEU
Levenshteing
RDK FTSt
MACC FTS*
Morgan FTS1
Validity
Alpaca
0.000
0.065
41.989
0.004
0.024
0.008
0.138
Buize
0.000
0.044
41,500
0.004
0.025
0.009
0.097
ChatGLM
0.000
0.183
40.008
0.050
0.100
0.044
0.108
LLaMa
0.000
0.020
42.002
0.001
0.002
0.001
0.039
Vicuna
0.000
0.057
41.690
0.007
0.016
0.006
0.059
Galactica
0.000
0.468
35.021
0.156
0.257
0.097
0,946
Text+Chem
0.239
0.782
20.413
0.705
0.789
0.652
0.762
Mol-Ins--Llama2
0.045
0.654
27.262
0.313
0.509
0.262
1.000
Mol-Ins--Llama..
0.883
0.863
0.708
Mol-LLama
0.440
0.912
17.120
0.724
0.859
0.665
1.000
HIGHT
0.037
0.869
23.759
0.59
0.394
0.34
0.993
EDT-Former 0.471
0.964
18.130
0.776
0.871
0.712
1.000
Table Results on the reagent prediction task from the Mol-Instructions benchmark. Models are fine-tuned and evaluated on the official splits, reporting 7 metrics as defined by the benchmark. The best (pink) and second-best (light pink) results are highlighted.
Model
Exactt
BLEU
Levenshtcin
RDK FTS
MACC FTS*
Morgan FTS
Validity
Alpaca
0.000
0.026
29.037
0.029
0.016
0.001
0.186
Baize
0.000
0.051
30.628
0.022
0.018
0.004
0.099
ChatGLM
0.000
0.019
29.169
0.017
0.006
0.002
0.074
LLaMa
0.000
0.003
28 28.040
0.037
0.001
0.001
0.001
Vicuma
0.000
0.010
27 948
0.038
0.002
0.001
0.007
Galactica
0.000
0.141
30,760
0.036
0.127
0.051
0.995
Text+Chem
0.000
0.225
49,323
0.039
0.186
0.052
0.313
Moi-Ins--Llama2
0.044
0.224
23.167
0.237
0.364
0.213
1.000
Mol-Ins.-Llama3.
0.101
0.648
18.326
0.412
0.521
0.375
1.000
Mol-LLama
0.495
49,230
0,521
HIGHT
0.462
28,970
0.314
0.275
EDT Former
0.145
0.650
46.950
0.464
0.531
0.431
1.000
4.4 ABLATION STUDIES
Comprehensive ablation experiments are conducted for question Q2-Q4, ranging from multimodal fusion graph patching strategies for Entropy- Guided Patching and connector design for Dynamic Query Transformer. All experiments use official splits with matched token budgets and hyperpa- rameter settings, App.D Additional ablation studies are in App.
Component Effects. Multimodal fusion and each core component are ablated on the four MoleculeQA tasks (Fig. 5). The full EDT-Former performs best across Structure, Source, Prop-
8

---

Published as a conference paper at ICLR 2026
WO Modality Fusion (-26.3%)
W/O Dynamic Query (-10.7%)
W/O Entrapy Patheing (-11.5%)
EDI-Former
10
N4.
10
72.3
m
665
67.0
57.8
Stret.
20
66.7
Src.
42 41.0
Prop,
42.0 40.8
01.7
App.
47.0
52.4
50
40
50
40
30
Figure 5: Ablation study of components on the MoleculeQA dataset. Accuracy is reported across four task types (Structure, Source, Property, and Application) when removing each component (modality fusion, Entropy- Guided Patching, or Dynamic Query Transformer).
Table 8: Ablations on patching strategies, Four methods are tested on the BBBP and Pampa. Accuracy (%) and Fl (subscript) are reported, with the top and second best highlighted.
Table 9: Effect of EDT- on different LLM backbones. Four novel LLMs are tested on the Molecule benchmark. Total accuracy and av- erage gain for each model are reported.
Methods
BBBP
Pampa
Avg. Drop
LLMs
LLM Only +EDT-Former Avg. Gain
Entropy
75.06:5.66
84.52v1.1
0%
Mistral-8B
28.00
55.63
+98.71%
BRICS 73.59s4.4
71.90s3.09
3.96%
Qwen3-8B
27.18
54.79
+101.58%
Random
68.90so.13
66.67/9.32
8.81%
Llama2-7B
38.37
65.89
+71.72%
None
39.6749.58
78.62%7.00
21.60%
Llama3. 1-BB
49.31
68.34
+38.59%
erty, and Application. In w/o Modality Fusion, the Dynamic Query Transformer is replaced by standard transformer blocks without graph-text-3D alignment; in w/o Dynamic Query, we keep the 2D/3D encoders and anchors but substitute the dynamic tokens with a fixed-length projection of graph embeddings; in w/o Entropy Patching, entropy sutrde segmentation is replaced by uniform 3- token SMILES patches. Removing multimodal fusion produces the largest degradation (26% avg.), confirming that combining text, graph, and 3D is critical for LLM comprehension. Disabling ei- ther Dynamic Query Transformer or Entropy-Guided Patching yields more than 10% average drops (10.7% and 11.5%), indicating that dynamic queries necessary to handle variable-sized graphs and capture more evidence than fixed-lene lendt fusion, while entropy-driven patches expose LLM- salient subgraphs for stronger structure-awan understanding (Q2, Q3).
Patching Strategies. We compare four segmentation methods on PAMPA/BBEP with matched splits, prompts, and query budgets: Entropy- Guided Patching (entropy peaks),
frag- ments (Jinsong et al. (2024), random patches (shufile from BRICS), and no patching. As shown in Tab. [] entropy-guiide patching yields the strongest results, BRICS trails slightly (chemically sound segmentation) while random patching degrades more (misses the true subgraph boundaries), and removing patching produces the largest drop, confirming that learned, data-driven boundaries are essential. Entropy peaks from a small next-atom transformer mark LLM-salient transitions, exposing true substructure boundaries the query interface; this data-driven segmentation likely generalizes beyond molecules and is necessary to realize multimodal fusion gains (Q2).
Backbone Sensitivity. On MoleculeQA with official splits, attaching our connector to Mistral, Qwen3, and Llama series models yields consistent improvements over the text-only backbones (Tab. Gains are larger for weaker backbones remain substantial across all four.
Efficiency Analysis. As shown in Tab. 10 rel- ative to LoRA finetuning, our strategy halves GPU memory and achieves ~3.5x faster train- ing per step under identical settings; full back- bone tuning is practically infeasible in set- ting. These support that efficient aligmment is achievable without updating the LLM (Q4).
Table 10: Estimated memory usage and training time per step for EDT-Former with Llama3.1 1-8B.
EDT-Former 1lama3.1-88 Mem (GB)
Time/Step (s)
Train
Frozen
37
0.26
Train
LoRA
77
0.93
Train
Train
>200

---

Published as a conference paper at ICLR 2026
Attention Score
Entropy
0.6
Graph Nodes SMIL ES Order
Polyamine backbone with pendant acetate groups
0.2
Anchor Q-Tokens
Dynamic Q-Tokens
Figure 6: Attention visualization for EDT-Former. Last-layer attention map for single molecule (N-63 atoms) with atoms in SMILES order along the vertical axis and query tokens (fixed-lenghh anchors followed by dynamic tokens) along the horizontal axis. The right panel shows the corre- sponding entropy signal for the SMILES sequence. Values indicate normalized attention scores.
Table 11:
Modality ablation on MoleculeQA. Four modality combinations are tested on four tasks, with the accuracies (%) reported. The top (pink) and second-best (lightpink) results are highlighted.
Modality
Structure
Source
Property
Application
Average
Total
Avg. Drop
Text/Graph/3D
74.55
72.39
50.71
48.58
61.56
68.34
0.00%
Text/Graph
70.84
68.72
49.22
46.51
58.82
65.11
4.73%
Tcxt/3D
66.50
64.77
42.17
42.02
53.87
60.48
11.50%
Text Only
66.46
60.98
40.35
36.40
51.05
58.77
14.00%
Fusion Benefits. In Tab. 11 four modality settings are evaluated on MoleculeQA. Text-only (SMILES) performs worst, underscoring the need for explicit structural features. Removing 3D causes a modest decline relative to full fusion, whereas removing 2D (graph) degrades substantially, indicating that graph information is essential while 3D provides complementary gains.
Attention Insights. The attention visualization in Fig. 6 reveals the complementary roles for the two query types. Fixed-length tokens behave modality anchors: their attention is diffuse and often spans multiple, unrelated patches, which useful for global alignment but unreliable for iso- lating substructures. In contrast, dynamic tokens align with the SMILES-ordered entropy segments and attend sharply to contiguous node ranges that correspond to chemically meaningful subgraphs (e.g., the polyamine backbone with pendant acetate groups highlighted). These patterns substantiate Q2 and Q3: multimodal fusion benefits when local structure is exposed, and the proposed novel- ties (Entropy- Guided Patching plus the Dynamic Query Transformer) provide substructure- aware interface that preserves local fidelity while anchors maintain global consistency.
5 CONCLUSION
EDT-Former is introduced as a method for aligning molecular graphs with large language models. Entropy-Guicde Guided Patching segments molecules entropy peaks to yield substructure-awant dynamic tokens, and Dynamic Query Transformer integrates these tokens with modality anchors to project into the LLM space, providing a stable interface without updating backbone parameters. Across standard benchmarks, EDT-Former delivers consistent gains over text-only and prior multimodal baselines while markedly reducing training cost relative to LLM tuning. Overall, EDT-Former offers chemically faithful, efficient graph-LLM alignment and general recipe for multimodal fusion that can extend to broader graph domains; code and processed data are released for reproducibility, and for current constraints and future directions, see Apps.
and
10

---

Published as a conference paper at ICLR 2026
6 ETHICS STATEMENT
This work does not involve human subjects, personally identifiable information, or non- public/proprietary datasets. All data are publicly available under their respective licenses, and no wet-lab or animal experiments were conducted. Although molecular AI can present dual-use risks (e.g., aiding the design of harmful compounds), tion learning and evaluation on standard benchmarks; we neither train nor release models intended generate or optimize novel toxic molecules. We will release code and models with terms of use that prohibit harmful or unlawful applications, All authors complied with institutional policies and responsible research practices.
contribution focuses multimodal representa-
REPRODUCIBILITY STATEMENT
This work is fully open-source under the MIT license. We release end-to-end resources to repro- duce every result in the paper, including: (i) complete training and evaluation code (with all ablation scripts), (ii) baseline and benchmark rumners with fixed seeds, (iii) processed pretraining and fine- tuning datasets with scripts to rebuild them from the original sources, (iv) model checkpoints, and (v) Dockerfile for ritwiss-rprooduccble environments. A single-entry script reproduces all main tables and even the figures; expected variances from stochastic training are reported in the repos- itory. No external services are required at evaluation time (offline scoring only). The repository is actively maintained and will serve as the foundation for subsequent extensions. All materials available via the anonymous GitHub link:
https: dtttps:/anonymonnt anonymonn 4open. science/r/
NV7-Former-34DD
As part of reproducibility, App. C.I and App. C.2 detail the implementation of our two novelties and the pre-training settings. The fine-tuning settings and parameters are provided in App. C.3 As well, App. D.2 documents the data processing for each dataset. Finally, App. D.3 reports the hyperparameter choices and model configurations for each benchmark.
8 ACKNOWLEDGEMENTS
This work was supported in part by the Canada Research Chairs Tier Program (CRC-20211- 00482), the Canadian Institutes of Health Research (PLL 185683, PJT 190272, PJT204042, CFA 205059), the Natural Sciences, Engineering Research Council of Canada (RGPIN-2021-04072. ALLRP 602759-24) and The Canada Foundation for Innovation (CFI) John R. Evans Leaders Fund (JELF) program (#43481).
REFERENCES
Walid Ahmad, Elana Simon, Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar. Chemberta-2: Towards chemical foundation models. In arXiv.org, 2022. doi: 10.48550/arXiv. 2209.01712.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Lue, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, A. Mensch, Katie Millican, Malcolm Reynolds, Roman Ring. Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei. Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, O. Vinyals, Andrew Zisserman, and K. Simonyan. Flamingo: a visual language model for fho- learning. In Neural Information Processing Systems, 2022.
Stella Biderman, Hailey Schoelkopf, Quentin G. Anthony, Herbie Bradley, Kyle O'Brien, Eric Hal- lahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, 2023. doi: 00.48550/a
11

---

Published as a conference paper at ICLR 2026
Andrés M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and P. Schwaller. Augmenting large language models with chemistry tools. In Nat. Mac. Intell. 2023. doi: 10. (038/s42256-024-0083
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, J. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Merbert-Voss,, Gretchen Krueger, T. Henighan, R. Child, A. Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Ma teusz Litwin, Scott Gray, Ben- jamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Sutskever, and Dario Amodei. Language models are few-shot learners. In Neural Information Processing Sys- tems, 2020.
Yongqiang Chen, Quanming Yao, Juzheng Zhang. James Cheng, and Yatao Bian. Hight: Hierarchi- cal graph tokenization for molecule-language alignment. In International Conference on Machine Learning (ICML) 2025, 2024.
Wenliang Dai, Junnan Li. Dongxu Li, A. M. H. Tiong. Junqi Zhao, Weisheng Wang, Boyang Albert Li, Pascale Fung, and Steven C. H. Hoi. Instructblip: Towards purpose vision-language models with instruction tuning. In Neural Information Processing Systems, 2023, doi: 10.48550/ arXiv.2305.0650
Ruiyi Fang, Liangjian Wen, Zhao Kang, and Jianzhuang Liu. Structure prenttrrrrpreeriinggg graph represen- tation learning. In 2022 IEEE International Conference on Data Mining (ICDM), pp. 927-932.
IEEE, 2022.
Ruiyi Fang, Bingheng Li, Zhao Kang. Qiuhao Zeng, Nima Hosseini Dashtbayaz, Ruizhi Pu, Boyu Wang, and Charles Ling. On the benefits of crribute-driven graph domain adaptation. The Thir- teenth International Conference on Learning Representations, 2025a.
Ruiyi Fang, Bingheng Li, Jingyu Zhao, Ruizhi Pu, Qiuhao Zeng, Gezheng Xu, Charles Ling, and Boyu Wang. Homophily enhanced graph domain adaptation. Forty-Second International Confer- ence on Machine Learning, 2025b.
Ruiyi Fang, Shuo Wang, Ruizhi Pu, Qiuhao Zeng, Hao Zheng, Ziyan Wang, Jiale Cai, Zhimin Mei. Song Tang, Charles Ling, and Boyu Wang. Graph domain adaptation via homophily-agnostic reconstructing structure. AAAI, 2026a.
Ruiyi Fang, Jingyu Zhao, Shuo Wang. Ruizhi Pu, Bingheng Li, Jiale Cai, Zhihao Li, Zihao Jing, Jian Zhu, Song Tang, Charles Ling, and Boyu Wang. Saga: Structural aggregation guided alignment with dynamic view and neighborhood order selection for multiview graph domain adaptation. The Fourteenth International Conference on Learning Representations, 2026b.
Yin Fang, Xiaozhuan Liang. Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. In International Conference on Learning Representations, 2023. doi: 10.48550/arXiv. 2306.08018.
Qiushi Feng, Danjo De Chavez, Jan Kihlberg, and Vasanthanathan Poongavanam. A membrane permeability database for nonpeptidic macrocycles. Scientific Data 12(1):10, 2025.
Thomas A Halgren. Merck molecular force field. i. basis, form, scope, parameterization, and per- formance of mmff94. Journal of computational chemistry, 17(5-6):40-511 -199, 1996.
Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovee, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development. arXiv preprint arXiv:2102.09548 2021.
Yunhui Jang, Jaehyung Kim, and Sungsoo Ahn. Chain-of-thoughts for molecular understanding. In arXiv.org, 2024. doi: 10448550/axiv.2210055661
Xiaohong Ji, Zhen Wang, Zhifeng Gao, Hang Zheng, Linfeng Zhang, Guolin Ke, and E. Weinan. Uni-mol2: Exploring molecular pretraining model at scale. In Neural Information Processing Systems, 2024. doi: 10.48550/arxiv.2446.. 14969.
12

---

Published as a conference paper at ICLR 2026
Zihao Jing, Yan Sun, Yan Yi Li, Sugitha Janarthanan, Alana Deng, and Pingzhao Hu. Structure- aware fusion with progressive injection for multimodal molecular representation learning. The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026.
Shao Jinsong, Jia Qifeng. Chen Xing, Yajie Hao, and Li Wang. Molecular fragmentation as a crucial step in the ai ai-based drug development pathway. Communications Chemistry, 7(1):20, 2024.
Dongki Kim, Wonbin Lee, and Sung Ju Hwang. Mol-Ilama: Towards general understanding of molecules in large molecular language model. In arXivorg, 2025. doi: 10.48550/arXiv25502..
13449.
Greg Landrum. Rdkit documentation. Release, 1(1-79):4, 2013.
Chanhui Lee, Yuheon Song, Yongjun Jeong, Hanbum Ko, Rodrigo Hormazabal, Sehui Han, Kyunghoon Bae, Sungbin Lim, and Sungwoong Kim. Mol-Ilm: Generalist molecular Ilm with improved graph utilization. ArXiv, abs/2502.02810, 2025.
Xiao Qing Lewell, Dunean B Judd, Stephen P Watson, and Michael M Hann. Recap retrosynthetic combinatorial analysis procedure: a powerful new technique for identifying privileged molecular fragments with useful applications in combinatorial chemistry. Journal of chemical information and computer sciences, 38(3):511-522, 1998.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301. 12597,, 2023.
Sihang Li, Zhiyuan Liu, Yancheng Luo, Xiang Wang, Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, and Qi Tian. Towards 3d molecule-text interpretation language models. In International Conference on Learning Representations, 2024. doi: 10. 00048550/rrXw..240..133
Yibo Li, Yuan Fang, Mengmei Zhang, and Chuan Shi. Advancing molecular graph-text pre-training via fine-grained alignment. In Proceedings of the 1t ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2. pp. 1589-1599) 2025.
Shengchao Liu, Weili Nie, Chengpeng Wang. Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Anima Anandkumar. Multi-modal molecule structure-text model for text- based retrieval and editing. In Nat. Mac. Intell., 2022. doi: 10.48 4445550002X..2222210775
Shengchao Liu, Weili Nie, Chengpeng Wang. Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar. Multi-modal molecule structure-text model for text-based retrieval and editing. Nature Machine Intelligence, 1(12):1447-1457, 2023.
Xingyu Lu, He Cao, Zijing Liu, Shengyuan Bai, Leqing Chen, Yuan Yao. Hai-Tao Zheng, and Yu Li. Moleculeqa: A dataset to evaluate factual accuracy in molecular comprehension. Conference Empirical Methods in Natural Language Processing, 2024. doi: 10.48850WaxXx.:2433.889999
Yi Luo, Kai Yang, Massimo Hong, Xingyi Liu, and Zaiqing Nie. Molfm: A multimodal molecular foundation model. In arXiv.org. 2023a.
Yi Luo, Jiahuan Zhang, Siqi Fan, Kai Yang, Yushuai Wu, Mu Qiao, and Zaiqing Nie. Biomedgpt: Open multimodal generative pre-trained transformer for biomedicine. In arXiv.org, 2023b. doi: 10.448550/axXiv22388.ooo 09442.
Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chiat Chen, Li Yuan. and Yonghong Tian. Prollama: A protein language model for multi-task protein language processing. In arXiv.org, 2024.
Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chian Chen, Yuan, and Yonghong Tian. Prollama: A protein large language model for multi-task protein language processing. IEEE Transactions on Artificial Intelligence, 2025.
MetaAI. The lama 3 herd of models. In arXivorg, 2024. doi: 10.4 10.48550WaxX1.2277.1133
13

---

Published as a conference paper at ICLR 2026
Mistral AL
Ministral-&B--instruut--- Vanistral-8B-Inttzuue 2410 2024. Model card, accessed 2025-09-23. 292-
https::/huggingffaee wwtp://www.....omsi....aaaa
OpenAl. Gpt-4 technical report, 2023. |nttps:/ hatee://peena..com/resaar//////
OpenAI. Introducing gpt-5. OpenAl Blog. 2025. URL bttps::/openai.. conpepaaccco//pt5///// Ac-
cessed August 7, 2025.
Artidoro Pagnoni, Ram Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtz- man, and Srinivasan Iyer. Byte latent transformer: Patches scale better than tokens. In Proceed- ings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), 2025. URL https:://///
ogy org/ 2025 acl llogg .433/
Jinyoung Park, Minseong Bae, Dohwan Ko, and Hyunwoo J. Kim. Llamo: Large language model- based molecular graph assistant. In Neural Information Processing Systems, 2024. doi: 10.48550/
arXiv.2411.000 1.00871.
Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang. Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan. Biot5+: Towards generalized biological understanding with iupac integration and mulliitask tuning. In Anmual Meeting of the Association for Computational Linguistics, 2024a.
doi: (0.48550/arxx1.2402.7788
Qizhi Pei, Lijun Wu, Kaiyuan Gao, Jinhua Zhu, and Rui Yan. 3d-molt5: Leveraging discrete struc- tural information for molecule-text modeling. In ICLR 2025, 2024b.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpaca: A strong, replicable instruction- following model. Stanford Center for Research on Foundation Models. https://rffm. stanford. ccd/2023/03/13aappee html, 3(6):7, 2023.
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint 1..0:22 2022.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, al. Llama: Open and efficient foundation lan- guage models. arXiv preprint arXiv:2302. 139771, 2023a.
Hugo Touvron et al. Llama 2: Open foundation and fine-tuned chat models. arXiv.org, 2023b.
BigScience Workshop. Bloom: A 176b parameeer open-access multilingual language model. In arXiv.org, 2022. doi: 10.48550//rXiv.221....
Fang Wu, Dragomir Radev, and Stan Z Li. Molformer: Motif-based transformer on 3d hetero- geneous molecular graphs In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 5312-5320, 2023.
Zhengin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learn- ing. Chemical science, 9(2): 2018.
Yingce Xia, Peiran Jin, Shufang Xie, Liang He, Chuan Cao, Renqian Luo, Guoqing Liu, Yue Wang, Zequn Liu, Yuun-Jyue Chen, al. Naturelm: Deciphering the language of nature for scientific discovery. arXiv e-prints, pp. arXiv-2502, 2025.
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. Baize: An open-source chat model with parametee--fffcient tuning on self-chat data. arXiv preprint urXiv:2304.01196, 2023.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 20Xiv:2505.09388. 2025.
Zheni Zeng, Yuan Yao, Zhiyuan Liu, and Maosong Sun. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals. Nature communications, 13(1):862. 2022.
14

---

Published as a conference paper at ICLR 2026
Juzheng Zhang, Yatao Bian, Yonggiang Chen, and Quamming Yao. Unimot: Unified molecule-text language model with discrete token representation. arXiv preprint urXiv:2408.00863, 2024.
Guojiang Zhao, Sihang Li, Zixiang Lu, Zheng Cheng, Haitao Lin, Lirong Wu, Hanchen Xia, Hengx- ing Cai, Wentao Guo, Hongshuai Wang, Mingjun Xu, Siyu Zhu, Guolin Ke, Linfeng Zhang, and Zhifeng Gao. Molreasoner: Toward effective and interpretable reasoning for molecular llms. In arXiv.org, 2025.
Haiteng Zhao, Shengchao Liu, Chang Ma, Hannan Xu, Jie Fu, Zhihong Deng, Lingpeng Kong, and Qi Liu. Gimlet: A unified graph-text model for instruction-based molecule zero-shot learning. In bioRxiv, 2023.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, E. Xing, Haotong Zhang. Joseph E. Gonzalez, and Ion Stoica. Judging Ilm-as-a-judge with mt-bench and chatbot arena. ArXiv, abs/2306.05685. 2023.
G. Zhou, Zhifeng Gao. Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: universal 3d molecular representation learning framework. In Inter- national Conference on Learning Representations, 2023.
15

---

Published as a conference paper at ICLR 2026
APPENDIX TABLE OF CONTENTS
Theoretical Analysis and Proofs
A.I Entropy Patching Analysis
A.2 Dynamic Query Analysis
B. Relationship with Previous Methods
B. From Representation to Multimodal Understanding
B.2 Relationship to Related Works
C. Implementation Details
C.I Entropy Guided Patching
C.2 Dynamic Query Transformer
C.3 EDT-Former Settings and Finetuning
D. Experiments Details
D.1 Benchmarks and Baselines
D.2 Data Process]
D.3
Evaluation Settings
D.4
Evaluation on Macrocycle Dataset-NPMiMPPD
D.5
Detailed Results of Property Prediction Tasks Prompt
D.6
Evaluation on Molecule Design and Open Questions from Mol- Instrnnntts
D.7 Natural Language Ability Analysis
D.8 Data Contamination Analysis
D.9 Rationale of Entropy Patching
D.10 Reproducibility
Extended Ablations
E.1
Impact of Query Token Length
Impact of Model Size
E.3
Analysis of Hyperparameter
E,4
Impact of Graph Encoders)
E.5
Impact of Different Global/L Token Budgets
E.6
Impact of Training of LLM Embedding Layer
E.7 7
Impact of Different Training Corpus
8
Inference time ablation of dynamic tokens and anchors
Hallucination Analysis
Limitations and Future Work
Limitations
E2 Future Work
G LLM Usage
A THEORETICAL ANALYSIS AND PROOFS
Scope and assumptions. The analyses App. A. and App. A.2 are intended as proof sketches under standard modeling assumptions (calibrated next-token predictors, piecewise-stationary gener- ation along SMILES order, and locally Lipschitz Transformer blocks on bounded inputs). We work with token-level surprisal e, tion). The results formalize why (i) segmentation at surprisal peaks aligns with change points and minimizes budgeted upper bound pooling/modeling loss, and (ii) anchor-aut augmented projec- tion yields stable, low-rank-exxe -ann--xxpesssive interface to frozen LLM. Constants and high-probabiliyg terms suppressed for clarity.
log p(a++1 (1=z) (and note that entropy refers to its expecta-
16

---

Published as a conference paper at ICLR 2026
A.1 ENTROPY PATCHING ANALYSIS
Setup. Let molecule be represented by a SMILES-ordered atom sequence (aI, ar) gen- erated by piecewise-stationary conditional distribution P*(a++1 @1:)). with unknown change points 1 = Ko < K1 < .... < RM = T such that for [Km--1, Km - 1] the law of ac+1 d1:t belongs stationary family Pm. trained next-atom predictor so outputs calibrated estimate po(a++1 @1::) Define the surprisal (negative log-likelihood)
e, 1o [1:),
1=
T
Entropy-guice patching places cut after indices in T (t : is local maximum of es refined by non-maximum suppression (window A) and a prominence threshold n, yielding segments Ss = T where 1 To < < TM T are the retained peaks.
{1: (1: Tk-
Objective. Given a token budget (implicit in A, we seek segmentation that (i) maximizes the information that segment tokens carry about the underlying structure, and (ii) minimizes within- segment heterogeneity that would be averaged out by pooling into a single token.
Let X E xx be frozen node embeddings and let π map SMILES indices to graph nodes. Each 3z)). Assumption (Piecewise stationarity). Within each true region [Km-1, fmm) the conditional dc+1 41a has constant entropy Hm e[ t (8m-1: kmm) and adjacent regions satisfy separation in total variation: TV(Pm, pm+1) 2 0.
segment induces node set S₁ = {t( t E S and a token 2 AvgPool([X;
Assumption (Calibration bounded error). Elee]] H (ac+1 (ax:)) S & uniformly in
Key quantity. For a candidate segmentation S = (S& M define the within-segment entropy
E(S) =
E[e.].
S
tES,
This upper-bounds both (a) the Bayes error of predicting local structural events within segments via Fano-type inequalities, and (b) the information lost by averaging embeddings inside a segment (see Lemma 2).
Lemma (Surprisal peaks mark change points). Under Assumptions 1-2, for any interior index I. that lies away from the nearest true change point, the expected discrete second difference satisfies E| E1+11 2e, et-1] S C1E -22 near a change point and 2 -CiE away from it, for constants C1, C2 > 0. Hence the probability that local maximum of e, occurs within O(1) of each change point tends =00
Sketch. At a change point, the one-step law jumps from Pm, Pm++. inducing kink in the log- likelihood path with magnitude controlled by TV Pm+1). Calibrated predictors track this kink up to e, yielding excess in e, locally. Away from changes, stationarity implies e, is a martingale difference with bounded variance, suppressing spurious peaks in expectation. The discrete curvature argument yields the stated bounds.
Lemma (Pooling loss is entropy-controlleds Let g be any I-Liischitt readout sets of node em- beddings (the self/cross-attentin layer consuming 2n is L--Lipschitz under standard assumptions). Then for each segment St.
E [ Sa)) 1((xx)154)11] C. v | St) C' VE(As)
where X is the mean in the segment and the final inequality follows because variation of embed- dings within segment controlled by variation of the local next-atom law, which is upper-bounded (up to constants) by the average surprisal in that segment.
Calibrated means B[1(A a) po results.
x) = consistency is not required, but strengthens the
17

---

Published as a conference paper at ICLR 2026
Proposition 1 (Peak cutting minimizes an upper bound loss). Among all segmentations with given minimum spacing (token budget), selecting cuts at the most prominent local maxima of et (with NMS window A) minimizes ((S) up to O(E), hence minimizes the stated upper bounds both Bayes error within segments and pooling-indee inggeed representation loss.
Sketch. Because Hm is (approximately) constant inside each stationary region, any cut placed within region increases (S) only by splitting a constant-entropy block, while cut placed at change point prevents averaging across different entropies Hn * m and thus strictly decreases E(S) by amount (2(6) (Assumption 1). Lemma guarantees that the largest local maxima of et concentrate at change points, so NMS over a spacing A selects (approximately) one cut per change region, which is optimal under fixed budget. Lemma 2 transfers the bound from & to the represen- tation loss after pooling.
Corollary (Adaptive token count matches structural complexity). Let K2 be the number of re- tained peaks after prominence and window A. Then [[[K.) is non-decreasing in the total varia- tion budget Em (V(Pm= p.m+1) and non-increasing in (A,7), yielding a token count that adapts to molecular complexity while remaining budget-controlled.
Robustness and determinism. The algorithm deterministic given (A, 7),, linear T for entropy and segmentation, and stable predictor noise: if æ < c (Assumption 2), the selected peaks coincide with truc change neighborhoods with high probability; otherwise NMS and prominence thresholding suppress spurious bumps that E(S) remains within ((e) of the optimal budgeted value.
Takeaway. Entropy--uided patching is principled change-point segmentation that (i) places boundaries where the conditional law changes (surprisal peaks), (ii) provably minimizes an entropy- based upper bound on within-segment error and pooling loss under a token budget. and (ii) adapts the number of dynamic tokens to structural complexity while remaining compute-efficient
A.2 DYNAMIC QUERY ANALYSIS
Setup. Let 7 {= be the variable-length substructure tokens from Entropy-Guided Patch- ing, A { be Q modality anchors (fixed, learnable, or lightly tuned), and let P E Rd Xdux be a connector that linearly projects tokens into the frozen LLM embedding space. The input to the LLM is the concatenated sequence X = [PA; PZ| E R(Q+K)xdux, Denote the first Trans- former block (self-attention + MLP) of the frozen LLM by f I((2+))xdux - (Q++))xduM, with attention maps parameterized by frozen matrices (We, WK, Wv) and output projection Wo- Subsequent blocks are identical copies with frozen parameters.
Assumption A1 (Lipschitz block), Each LLM block is L-Lipschitz on a compact input set X: f(Y) s x-| (this holds under standard
assumptions). Assumption A2 (Anchor coverage). Let SA span W x PA) C Rd be the key subspace induced by anchors. There exists amin > such that the minimum singular value of [WRPA] Z mmin and span(Wq P = SA as well (anchors expose stable keylquery frame).
Assumption A3 (Bounded projection error). Let Z* IR KxduM be an (unknown) ideal em- bedding that would produce the desired frozen-LLM behavior. The learned connector satisfies [PZ - |1 < on the data manifold.
Stability via anchors. Consider attention logits in the first block: S = = XWoWW K- \/ Anchors contribute structured rows/columns SAA and SaZ that do not depend on Z's length or order. Because A2 ensures a well-conditioned anchor subspace SA, the Jacobian of attention w.r.t PZ is bounded, which yields an input-output Lipschitz bound uniform in K.
Lemma (Anchor-conditioneed Lipschitz stability). Under A1-A2, there exists C(L, minn || || II. [|Wr:l)) such that for any two token sets Zi, Zz (possibly different lengths) and the same anchors A.
[[PAA; PZ))) J(IPA: S ClIP7 P7z||
18

---

Published as a conference paper at ICLR 2026
Hence the per-block perturbation is controlled by the connector-space distance, independent of K.
Sketch. Write the attention output as a composition of (i) bilinear logits (oWo(W' K and (ii) softmax-weighted value mixing. The anchor block PA produces keyqueery frame whose spectrum is bounded below by min (A2), preventing ill-conditioning in the softmax sensitivity. Combine with A1 and standard Jacobian bounds for softmax obtain the stated Lipschitz constant
Expressivity vs. LoRA at the input. Linearizing the first block at an operating point Xo [PA; Pool gives f(X) = f(Xo) + Jo(X Xo). where Jo is the Jacobian. A rank-r LoRA update the input embedding (E + E + A lank(A)) (E) S r) induces an output shift Jn of rank at most If P is factored as P = UV with rank(P) S r, then P7 can realize equivalent family of rank-r shifts in the first-block input, matching what a LoRA adapter would add at that boundary.
Proposition 2 (Connector-only can emulate low-rank input adaptation). Fix anchors A and lin- earization point Xo. For any rank-r update AE to the inpul embeddings, there exists rank-r connector P UVT and a perturbution AZ such that Jol [0 AE]) = Jo([0; P&Z)). Hence, in the first-order regime, connector-ont training with rank-r P is expressive rank-r LoRA at the input boundary.
Sketch. Let Ja define linear map on inputs; match images by choosing PAZ AE in the column space that Jo uses. Factor P with rank I and select AZ in the corresponding r-dimensional coordinate system to reproduce Jg.E.
Information preservation. Let be the (frozen) LLM map from the first block the layer used by the cross-attention interface (or to logits for analysis). Define the task-relevant mutual informa- tion / (2; ([PA; PZ]).. Because f is L-Lipschitz and the connector error is (A3), data processing and stability imply only small information loss.
Lemma 4 (Lower bound retained information). Under Al-A3 and mild regularity (bounded support and densities), there exists & >> O such that
I(Z: O(P: PZ)) 2 ((::6((PA Q..
Thus, if P approximates the ideal embedding within downstream information loss is O(E).
Sketch. Combine Lemma 3 with standard continuity bounds for mutual information under Lips- chitz perturbations (via Wasserstein stability Pinsker-type inequalities). The a constant absorbs
Lipschitz and measure-regularity terms.
Budget awareness and interference control. Let l be the orthogonal projector onto span PA} in the LLM embedding space. Adding orthogonality penalty (PZ)]2 ensures that content tokens PZ occupy a subspace complementary to anchors, reducing cross-attention interference and concentrating gradients on task-relevant directions.
Proposition 3 (Conditioning and interference). If training enforces (approximately) Il and normalizes PA, then the attention Gram matrix on X has condition number bounded by constant depending only on anchors. Consequently, gradient norms for P are well-conditioned and do not deteriorate with token count K.
Sketch. Under the orthogonality constraint, the mixed Gram blocks between anchors and content vanish at first order, yielding block-structured Gram with bounded spectrum the anchor block (A2) and controlled spectrum on the content block via normalization and Lipschitzness, hence a uniform bound on the condition number.
19

---

Published as a conference paper at ICLR 2026
Takeaways. (i) Stability; anchors provide well-conditioned key/query frame that makes the frozen LLM locally Lipschitz in the connector space, uniformly in token count (Lemma 3). (ii) Ex- pressivity: low-rank connector emulate first-order effects of low-rank (LoRA) input adapter (Proposition 2 explaining strong performance without tuning the backbone. (iii) Information reten- tion: if P approximates ideal embedding, the downstream information loss is O(e) (Lemma 4
(iv) Good conditioning: orthogonalizing content against anchors controls attention conditioning and prevents interference (Proposition 3
Overall, Dynamic Query Transformer yields stable, expressive, and budget-aware interface to frozen LLMs, complementing Entropy-Guie -uidded Patching's 's adaptive tokenization.
B EXTENDED RELATED WORK
In addition to the discussion in Sec. 2 this section provides further analysis of how our approach relates to prior efforts. Specifically, we highlight the differences between our Dynamic Query Trans- former and fixed-length Q-Former-style bridges, as well as connections to entropy- works and recent multimodal molecular language models. This extended review situates model within the broader landscape and clarifies novel contributions.
B. FROM REPRESENTATION TO MULTIMODAL UNDERSTANDING
Representation models vs. language models. Early molecular representation learning focused embedding models that map graphs sequences to vector spaces for downstream prediction, e.g., MolFormer (Wu et al., 2023). Uni-Mol/Uni-Mooe
(Zhou et al., 2023 Ji et al., 2024). and ChemBERTa-2 (Ahmad al.) (022). These encoders excel property classification and regression as they optimize contrastive and masked-token training over large chemical corpora, yielding stable, geometry-aware embeddings for downstream heads. However, they do not natively support natural- language generation (e.g. UniMoT (Zhang et al. (0244) or multi-hop reasoning (Jang et al.) 2024); in these settings, evidence remains implicit in continuous vectors, and exact numeric targets are better served by calibrated regressors than autoregressive tokens.
From generation to reasoning with LLMs. LLM--ased molecular systems provide a natural- language interface for explanation, instruction following, and tool use, enabling generative descrip- tions and stepwise reasoning (Fang et al. 2023 Zhao et al. 2023: Zhang et al. 2024). Yet vanilla LLMs struggle with exact property prediction due the mismatch between discrete token likeli- hoods and calibrated real-valued targets, unit/scale sensitivity, and exposure bias during numeric generation; recent work addresses these issues with instruction data, cuol-augmented agents, and -hain-of-thought methods tailored to molecular structure (Kim et al., [025 Bran et al.) [023 Jang al. 2024). Overall effectiveness hinges on how molecular structure exposed the language mooel--hetteer via tokenized geometry, graph-aware connectors, substructure tokens.
Multimodal fusion for molecular generative modeling. Beyond SMILES-only models (e.g..
KV-PLM (Zeng al. 2022)), recent work fuses SMILES, molecular graphs, and 3D conformers with LLMs. Instruction-tuuee systems such as GIMLET (Zhao et al.) 2023) show that paired graph-
text supervision imparts basic molecular understanding: larger frameworks (c.g.. MoleculeSTM (Liu ct al. 2023), MolFM (Luo et al. 2023a). BioT5+ (Pei et al.) (224a). ProLLaMA (Lv et al. (025)) broaden modality and biomedical knowledge coverage; and 3D-aware models (e-g.. 3D-Mol (Pei
et al. 2024b)) introduce coarse geometry for conditional generation. More recent gLM con-
nectors (e.g., Mol-LLM (Lee et al. 2025), Mol-LLaMA (Kim et al. 2025)), and UniMolT (Zhang al. 2024) adapt the OLIP-2/QQForrme eL al. 2023) paradigm with fixed-length, learnable query tokens. Although effective for alignment, fixed- length fusion compresses stereochemistry and sub- structures, resulting in structural information loss that worsens with increasing molecular size and conformer diversity; this motivates the development of dynamic and substructure-awar interfaces that present structures more faithfully to the L.M.M
20

---

Published as a conference paper at ICLR 2026
Standard Q-Former Based Model
Q-Former
Q. Dlescribe the fiactional groups present the follewiing moiecule...
Muliinodal Ensodiass
Biensing Modile
Large Language Model
Self-Ara
A: The molecale compter on ester group carbuay acid group benzene ring...
Faamabl Yokees
Novel EDT-Former Based Model
Encoding
FITT-Former
Strre
Giep
Node level Mege
Q: Deserill der Samberdee
fffnctional groupe...
3B
Cross Arta
Crose Atm
Prosttsa
Large Language Model
Self-Attn
Self-Attn
Qu
The muriec rentains on exterr group curbosylic acid group. and aromatic beanon ring.
Mabillo Anchar
Smuckre-Aver
Eatropy-Basce Basee
Dyeance Lungh
Paach Pral
Figure 7: Architecture comparison between Q-Former-basee model and EDT-Former. (a) Standard Q-Former connector that uses fixed-length learnable query tokens as modality anchors. (b) Novel EDT-Former connector that combines fixed-length tokens and entropy-guided structure-aware dy- namic query tokens.
B.2 RELATIONSHIP TO RELATED WORKS
Relationship to Q-Former. BLIP-2 introduces a lightweight Querying Transformer (Q-Former) that uses fixed set of learnable query tokens to extract visual evidence from frozen image en- coder via cross-attention, then projects the resulting tokens frozen LLM enabling efficient vision-language alignment without updating the backbones (Li et al. 2023). The Q-Former is pretrained with vision-language objectives (e.g., contrastive/alignmeet
ensure its queries attend to informative regions and produce I.L.M generative stages) to embeddings. Sub- sequent systems (e.g., InstructBLIP (Dai et al. 2023)) keep the same mechanism but condition the Q-Former textual instructions, improving instruction following while retaining the fixed query budget. Related adapters such as Flamingo's ' (Alayrac et al., 2022 Perceiver- Reeeevee-RReammple also distill high-dimensional visual features small, fixed-length token for frozen LM.
As shown in Fig. 7 Dynamic Query Transformer differs in two aspects. 1) Modality. Molecules are variable-sized graphs (and conformers), where structural fidelity hinges preserving substructures (stereochemistry. functional groups), not just salient image regions. 2) Interface. Instead fixed query budget. EDT-Former upgrades the Q-Former's fixed queries with entropy-guided dynamic queries. Concretely, Entropy-Guided Patching discovers entropy-peak-basee node embeddings into dynamic tokens, and Dynamic Query Transformer refines these tokens (with small set of anchors) via htt////rrossatttiiioooo before projection to the LLM space. This design pre- serves Q-Former- style efficiency (frozen backbones, lightweight adapter) while improving graph- specific fidelity by matching the number and placement of queries to molecular structure, thereby enabling multimodal molecular models without fine-tuning the LLM backbone (excluding em- bedding layer). In short, we adopt the Q-Former rationale- cereee-rreeeeee encoders bridged by compact adapter but move beyond fixed-length queries to a dynamic structure- aligned interface tailored to graphs.
segments and pools
Relationship to BLT (Byte Latent Transformer). BLT groups raw bytes into dynamic variable- length patches using the entropy of the next byte, so that higher-entropy regions receive more com- pute; latent transformer then operates over these patches and projects to language model, yielding tokenizer- free efficiency and robustness (Pagnoni et al., (025). Our approach draws inspiration from the entropy-driven segmentation principle, but targets different modality and objective: molecules
21

---

Published as a conference paper at ICLR 2026
variable-sized graphs where fidelity depends preserving substructures and stereochemistry. Concretely, we estimate next-atom uncertainty over SMILES to place peak-based cut points, map segments to graph nodes, and pool node embeddings into dynamic substructure tokens. These tokens are integrated (with a small set of anchors) via self-/cross-attention and projected to a frozen LLM,, aligning query number and placement to molecular structure. Thus, while BLT's entropy patching motivates adaptive granularity, our entropy definition, boundary rule, and graph-aware interface are tailored structural chemistry rather than byte-level text.
Comparison to other molecular LLMs. (1) Mol-Instructions (Fang et al. 2023) Mol- Instructions establishes large-scale instruction data for molecules and fine-tunes general LLMs (c.g., Llama-2/3 (Touvron et al., 2023b MetaAl 2024) from SMILES inputs demonstrate the value of instruction supervision for captioning and property-related responses. We adopt its benchmarks and reported baselines for our evaluations on molecular description and property prediction. While effective as single-modality (text/MMILES) approach, it does not expose explicit structural signals (graphs/3D), which limits structure aware reasoning beyond what can be inferred from tokenized strings. (2) Mol-LLaMA (Kim et al. 2025). Mol-LLaMA bridges molecular structure and language using O-Former-style connector: a fixed set of learnable query tokens cross-attends to strong encoders and conditions an L.M.. This design proves the feasibility of graph-ILLM alignment but inherits the fixed-length bottleneck, which compresses substructures and stereochemistry as molec- ular size and conformer diversity grow. In contrast, our method retains the lightweight adapter philosophy while replacing fixed queries with dynamic entropy-guided substructure tokens refined by dynamic query transformer. For completeness, we employ recognized 2D/3D encoders (e.g., MoleculeSTM (Liu et al., (202)), Uni-Mol (Zhou et al. (2023) and comparable processed training data when reproducing baselines; these choices standardize comparisons but are orthogonal to the core novelty.
Other graph-text alignment works. Recent frameworks such as MoleculeSTM (Liu et al. 2023) and MolTex (Li al. (0255) learn new multimodal encoders from scratch via large-scale scale struc- ture-text contrastive training and motif-level masking curated cooleuueecaapion pairs, targeting strong standalone representations for retrieval and editing without relying on a frozen LLM. con- trast, EDT-Former adopts connector-only paradigm: we freeze both the molecular encoder and the LLM backbone (excluding the embedding layer), and learn a lightweight bridge that uses entropy- guided dynamic SMILES patches plus few anchors to adapt the token budget to molecular com- plexity. These approaches are complementary--suee pretrained encoders could serve as stronger molecular backbones, while EDT-Former focuses on efficiently aligning them with instruction-tuned LLMs for QA, property prediction, and reaction-centri generation.
Inspiration from graph and fusion embedding models. Our previous work MuMo <Jing et al. 2026) is multimodal embedding model for sequences, graphs, and 3D structures, evaluated across a comprehensive suite of molecular tasks. We also drew broader insights on graph modeling and transfer techniques from graph-related works: (Fang al. 2022), (Fang et al. 2025a), (Fang al., 2025b), (Fang et al. 2026a), (Fang et al. 2026b).
C IMPLEMENTATION DETAILS
This section details the implementation for reproducibility. It begins with the entropy-guided patch- ing algorithm, including entropy computation, peak detection, the next-token predictor, and a brief rationale recap. Attention computations for integrating dynamic tokens with anchors are then for- malized, Configurations of EDT-Former, covering both graph encoders and LLM backbones, are described next. The section concludes with pretraining settings (objectives, prompts, and hyperpa- rameters) and finetuning protocols (sctups, prompts, and representative inference examples).
C.I ENTROPY-GUIDED PATCHING
Algorithm of Entropy-Guided Patching. Algorithm 2 segments a SMILES sequence into dy- namic substructures using entropy signal from lightweight next-atom predictor (NAP). For
22

---

Published as a conference paper at ICLR 2026
Table 12: Configurations and Training settings for the Next-Atom Prediction (NAP) model, archi-
tecture, hyperparameters, together with the resulting compute footprint and parameter count.
Configurations
Pre- Fre-Training
Item
Value
Item
Value
Vocabulary size
39
Learning rate
1 x 10
Model architecture
GPT-2LMMeed
Batch size
64
Maximum position embeddings
1024
Training epochs
1
Context window
512
Training steps
4,929
Transformer layers
2
Weight decay
0.01
Attention heads per layer
2
Total FLOPs
7.5395 x 10
Hidden size (embedding dimension)
128
Model size
0.538M
Activation function
GELU
Dataset
Pubchem
Dropout residual connections
0.10
Max Length
128
Dropout token/position embeddings
0.10
Concat Data
True
Dropout attention
0.10
Precision
FP16
LayerNorm epsilon
x 10-5
Objective
NAP
Table 13: Molecular atom vocabulary from SMILES grouped by category for the NAP (Next-Atom Prediction) model, with per-category counts (total tokens 39, including 3 special tokens).
Category
Symbols
Count
Special tokens
<pad>, <bos>, <eos>
3
Halogens
Cl, Br, I
4
Noble gases
He, Ne, Ar, Kr, Xe
5
Alkali metals
Li, Na
2
Alkaline earth metals
Mg, Ca
2
Transition metals
Fe, Co, Ni, Cu, Zn, Ag, Au, Pd, Pt, Mn, Hg
11
Post-transition metals
A1, Sn
2
Metalloids
B, Si, Sb, Te
4
Other nonmetals
C, N. O. P,
6
Total
39
each position, the predictor provides the probability of the ground-truth next atom; its negative log- probability defines the entropy trace. Local maxima are detected with prominence threshold and non-maximum suppression, and cuts are placed after the retained peaks. Segments are mapped from SMILES indices graph nodes and average-pooled over frozen node embeddings to form dy- namic substructure tokens. The procedure is simple, deterministic given (A.7), linear in sequence length for entropy and segmentation, and produces structure-aware tokens aligned with molecular complexity.
The NAP model configuration. As summarized in Tab. 12 the NAP model adopts standard transformer architecture with maximum input context of 512 tokens. This context window bounds the effective sequence length used during training and inference. The SMILES atom vocabulary shows in Tab. [3) comprises 39 tokens and spans major chemical groups-halogens, noble gases, metals, metalloids, other nonmetals-
alkali/alkalinee earth metals, counstiion/posttrrnssit providing broad elemental coverage for next-atom prediction.
Pre-training of NAP. We train the NAP model with standard autoregressive next-token prediction over SMILES. Given tokenized sequence #1:7 (prepended with (bos) and optionally terminated by (ecs)), a causal Transformer parameterized by B defines the left-to-right factorization
Pa (a( =
po ( (2x))
(8)
1=1
23

---

Published as a conference paper at ICLR 2026
Algorithm 2: Entropy-Guided Patching
Input SMILES atom sequence (a1,..., ar); trained next-atom predictor føi node embeddings X E Nxx (frozen graph encoder); map (SMILES (HILLE))gaaph); NMS window A; prominence ? SMILES segments (Sk 15
Output: Dynamic tokens (=x 1
Entropy computation:
for i - to To do
L f
// logits over atom vocab // prob. of ground-truth next atom // information content (nats); O(T) total
+ softmax((1)) (+4))
e, log Ps
Peak detection (local maxima prominence):
T. 0 for t - 2 to do
// candidate split indices
Msx <- (et-1 < et) (ex e+1); prominent + (e: - f(e + Ex-1) 2 7)
// strict peak // remove shallow bumps
if LsLLooalMax A prominent then
L T. + U (t);
// keep peak t
non-maximum suppression (window A);; Sort T. by descending &, to get (1(2), (22) ((2),...) O log (T.))
// highest entropy firsty
T
// final peak set // suppress neighbors within A
foreach E (t(1), ((2) do
Vu T. lu v
A
then
T - TU U fut
Segmentation (eut after peaks);;
Set 1 = < <... TM-1 = T with ( - =T;
// deterministic cuts for k 1 to M do
S (1 - 51 <I<T):
// SMILES index segment // map to graph nodes
S, (1( 8
Pooling to dynamic tokens (avg-pool):: for k + 1 to M do
X;:iE S &x E ERd; // substructure token; linear in Fk < AvgPool (x,
S
return { (
// dynamic structure-aware tokens
We optimize the negative log-likelihood (cross-cntropy) under teacher forcing, averaged over non- padding tokens:
C(0) ΣT
m. Σ me log x<t).
where m, (0.1) masks out (pad) (and any positions beyond the context window). In practice, logits are computed over the 39-token vocabulary; optional label smoothing can applied by re- placing the one-hot targets with smoothed distribution. During inference, sample (or decode greedily) from pe(- x until (eos) or the 512-token context limit is reached.
Computing costs analysis of NAP model. Using the settings in Tab. 12 on single RTX 3090, end-to-end training lands in the "single-digit minutes" range (well below 0.1 GPU-hours for a full pass), the cost is effectively negligible. The model is tiny (* 0.54 M parameters) and can also be trained comfortably on a modern multi-core CPU within an hour, avoiding the need for a GPU altogether. Compared with typical 8B 8B-parmmeter model, this NAP setup z 150x smaller and
24

---

Published as a conference paper at ICLR 2026
Table 14: Configurations and pre-training settings of Dynamic Query Transformer. The model configuration is based on SciBERT and trained from scratch.
Item
Value
Item
Value
Base config
SciBERT
Embedding dimension
512
Max dynamic queries
64
Anchor queries
16
Max input tokens
594
Batch size
48
Attention heads
8
Transformer layers
Training epochs
10
Total steps
15,599
Precision
bf16-mixed
Scheduler
cosine
Initial learning rate
1.0 10
Minimum learning rate
1.0
10-5
Warmup learning rate
1.0 x 10-6
Warmup steps
500
Weight decay
0.05
Temperature
0.10
Top-A
50
Top-p
1.0
Repetition penalty
1.0
Accumulate grad batches
1000x faster for both training and inference, it does increase the training or inference budget.
Entropy peaks from the NAP predictor identify structural transition points in SMILES; cutting at these peaks yields data-driven substructures and dynamic tokens that preserve locality and align cleanly with the language stream-simple, rule-free, and robust under a frozen backbone. The pipeline adds negligible overhead and does not materially affect training or inference costs.
C.2 DYNAMIC QUERY TRANSFORMER
Modeling Details with Anchors and Dynamic Tokens. We keep standard Transformer machinery and only specify the routed, two-stream updates. Let Q E R (anchors) and Z Rd (dynamic tokens).
(1) = (0) + GelfAttnax(Q 2 fix,sa
(10)
z(0)
z + SelfAttnz(Q z00).
(11)
sa
With graph node embeddings X E R Nxd xd each stream retrieves evidence independently:
2 = 9 (f)
+ CrossAttnn,: fix,sa X).
(12)
fix
(rossAttn<ZZi0... x).
(13)
shared FFN is applied to each stream separately:
(E+1)
5 fi
z(++)) 200) + FFN (()).
(14)
After L layers, project to the LLM space and concatenate to form the conditioning sequence:
U =
[ fix Projj:
((4) proj]
R Mk++M)xdux..
(15)
Configurations and pre-training settings. We pretrain the Dynamic Query Transformer bridge with fixed anchor/dynamic query budget, moderate-depth Transformer, and standard mixed- precision schedule with warmup, cosine decay, and weight decay; full hyperparameters and decod- ing settings are reported in Tab. 14
Pre-training objectives. The Dynamic Query Transformer is pre-trained with three complementary objectives Mol-LlamaaIInstee dataset (Kim et al. 2025): (1) cross-modal contrastive alignment to pull together representations of the same molecule across modalities and push apart different make fixed anchors decode the modality identity, enforcing
molecules; (2)
25

---

Published as a conference paper at ICLR 2026
shared semantic interface; and (3) masked substructure reconstruction to teach dynamic tokens to preserve fine grained structural content.
(1) Cross-modal contrastive loss. For mini-batch B, and modalities and m', we use InfoNCE objective
expl (apsim(2(ttm) (2(6.m), ( 2
Ccontrast
log
(16)
B
2jeB jEB exp (sim (2 2(imm())/))
(i.m)EB
where 26.m) E 12d is the pooled query embedding of molecule in modality m, sim is cosine similarity, and is temperature.
(2) mmachiny loss. Fixed-length anchors Qäx must predict the modality identity via an M-way classification:
Lmatch
1
Σ
CE(W z(0, m), m
(17)
B
(im))
where z(f) is the average of the anchor outputs for molecule i, W are classifier parameters, and CF. is cross-entropy.
(3) Masked substructure reconstruction loss. Dynamic queries Qaya are trained to recover masked fragments within each modality:
Leconn
Σ
CF( (z(im)), masked maskee_.tokenns..)))
(18)
B M
(im)EB
where ((m) is a modality-speciffe decoder and masked (i.m) are the targets.
The total objective is a weighted sum:
Lental = Å1 Loontrast: + X2 Lmatch + Aa Precon:
A A2, 13 O.
(19)
C.3 EDT-FORMER SETTINGS AND FINETUNING
After pre-training the Dynamic Query Transformer, the next stage is finetuning this connector with frozen LLMs.
Selection of LLMs.. Dynamic Query Transformer bridge, we fine- tune only the connector jointly with graph en- coders and the LLM, while keeping both the encoders and the LLM frozen. We screen several 7-8B backbones, including Mistral- 8B (Mistral Al 2024)) Qwen3-8B (Yang et al.] 2025), Llama2-7B (Touurron et 2023b), and Llama3. (MetaAl 2024)-covering strong base and caasoning-orienter models. As shown
g After pre-training the
Table 15: Fine-tuning dataset. Four types of in- struction data from Mol-Llama-Instuene the data types and amounts are listed.
Category
Amount
Detailed Structural Descriptions
77,239
Cenuctue-to-Chemiie Features
73,712
Structure-to-Biolodd Features
73,645
Comprehensive Conversations 60,147
in Tab. 9 Llama3. yields consistent performance on the MoleculeQA (Lu et al. 2024) bench-
mark. We therefore adopt Llama3.1 1-8B as the default backbone for subsequent experiments, as provides the strongest raw molecular understanding and the best absolute performance once paired with our connector. Notably, EDT-Former delivers consistent gains across LLM backbones, un- derscoring the effectiveness of our connector design and its robustness to different architectures. For the frozen backbone, we freeze the main layers and retain the embedding layers trainable to ensure a stable convergence process, the graph tokens from the Dynamic Query Transformer mapped into LLM's embedding space.
Finetune configuration and settings. The graph encoders and LLM backbone the default settings released by the author (Zhou et al. 2023 Liu et al. 2023). The training settings of Dynamic Query Transformer list in the T6b
Training dataset and prompt examples. As summarized in Tab. 15 our fine-tuning corpus from Mol-I Inst--uuut comprises four instruction types: (i) Detailed Structural Descriptions, that
26

---

Published as a conference paper at ICLR 2026
Table 16: Fine-tuning settings for EDT-Former with frozen Llama-3.1-8B backbone.
Item
Value
Item
Value
Precision
bfl6-mixed
Epochs
2
Accumulate_gey _ereilatteerd____eee batchess
8
Weight decay
0.05
Initial learning rate
1.0 x 10-4
Minimum learning rate
5.0 x 10
Warmup learning
1.0 10-6
Warmup steps
1,000
Scheduler
l wwrmup.cosinee.... _lr
Temperature
0.10
LoRA rank (r)
8
LoRA alpha
32
LoRA dropout
0.10
Trainable parameters
213M
Total parameters
8.3B
System: "You are a helpful assistant specializing in chemistry and biology. The instruction that describes a task is given, paired with molccules. Write response that appropriatcly completes the request."
Structural Descriptions
User: "Can you tell me about the basic structure of the given molecule? <mol>."
"The given molecule complex carbohydrate polysacchuride derivative that contains several monosaccharide units with multiple functional groups modifications. un'nl. **Core Structure**; The backbone based on cyclic carbohydrates. Specifically,...
Assistant:
Straetuneto-CCemiical
User: "Molecule <mol> mnoff/nnOffer thorough amalysis of the chemical characteristics of the compound."
Assistant: "The given molecule has a structure that combines several functional groups, imparting unique chemical properties to it. Centrally, it contains propenone linkage, which features carbon-carbon double bond conjugated with carbonyl group. This conjugated system not only...
Structunetto-Billoiica
"Offer an in-depth discussion of the biological traits of the molecule, specifically highlighting the relationship between the core parts of the molecule and its properties. 'nMolecule <mol>."
User:
Assistant: given molecule a complex natural product found in Illicium verum, commonly known as star anisc. The biological properties of this molecule are influenced by its structural features, which include several characteristic functional groups and cttrrructturs..nniiiiiiii Tetracyclic Core Structure:
Comprehensive Conversations
User:
"Can you tell me about the basic structure of the given pooeecule <mol>."
Assistant: "Certainly! The given molecule is lactone, more specifically, five-membered cyclic ester. It 3-position the ring and methyl groups the 5-position.'n
includes methylene group attached
s "What are the key functional groups present in the given molecule?"
Assistant: "The key functional group the given molecule is the lactone, which eyelic ester. In addition, there's an exocyclic methylene group at the 3-position.'n"
Figure 8: Prompt setup and instruction data examples of EDT-Former in the finetuning stage with backbone frozen. One instance is shown for each data type; the comprehensive conversation cate- gory contains multiple rounds (exceeding ten).
elicit atomm-substruccure-leeell narratives of a molecule; (ii) Structuretto-Chemical Features, link- ing structure to physicochemical properties; (iii) Structure-to-Biolgge Features, covering bioac- wvity/ADME-sstye attributes; and (iv) Comprehensive Conversations for multi-turn reasoning and explanation. The dataset contains 284.743 instances in total, providing balanced coverage of descrip- tive, analytic, and conversational skills. Fig. shows the prompts and examples of the finetuning data.
27

---

Published as a conference paper at ICLR 2026
D EXPERIMENTS DETAILS
Here, we provide additional experimental details complement the main results. This section first describes the benchmarks and baseline systems used for comparison, followed by the evaluation setup (splits, prompts, metrics, and decoding). We then include representative response examples to illustrate typical model behavior, and then we report our data-contamination checks and screening procedures. Finally, we document reproducibility essentias- and other necessary information to enable faithful re-runs.
D.I BENCHMARKS AND BASELINES
BENCHMARKS
MoleculeQA. A large-scale multiple-choice QA benchmark for molecular factuality, where each instance pairs human-writte ques- tion with one correct option and three dis- tractors from authoritative molecule descrip- tions (Lu et al., (024). It covers four aspects: Structure (configuration, functional groups, backbone), Source (isolation, discov- ery/derivation, metabolites), Property (physic- ochemica//iooloical properties, safety, mecha- nisms), and Application (therapeutic/chemical uses, approvals, ascarch/agricultural agents). Molecules are split by Bemis-Murcko scaffolds into train/dev/test sets of QA pairs as total of 61,574. Aspect-wise counts are reported in Tab. 17
Table 17: MoleculeQA split counts using Bemis- Mureko scaffold-based splitting at the molecule level to avoid scaffold overlap across splits.
Category
Train Valid Test
Overall
49,993
5,795 5,786
Structure
32,176 / 3,314 / 3,113
Property
4,838 698 / 731
Application
1,917
558 / 599
Source
11,062 1,225 / 1,343
Hol-Instructions. Mol-Instructions Fang et al. 2023) is a large-scale instruction-tuning corpus for the biomolecular domain in instruction ressoonse format for LLM supervision. Each example couples a natural-languaae prompt with target text for molecule-centric skills. We focus on seven tasks: six molecule-oriented tasks and one open question task. The corpus spans over two million instructions and includes the captioning and property instruction sets used in our fine-tuning.
Molecular Property Prediction. We use ten focusse datasets from Therapeutics Data Commons (TDC) (Huang et al.] (22TT) and MoleculeNet (Wu et al., 2018). For instance, BBBP (Blood-Brain Barrier Penetration) a binary classification task predicting compound's ability to cross the blood-brain barrier. PAMPA (Parallel Artificial Membrane Permeability Assay) predicts passive membrane permeability from an in vitro surrogate of oral absorption. Both datasets are standardized in TDC for model development and fair comparison.
BASELINES
General Language Models. We compare against strong general LLMs spanning proprietary and open releases: GPT-4o (OpenAT, 2023) and GPT-5 (OpenAl 2025 as OpenAl's flagship multi- modal and next-generation reasoning models. From Meta, we include Llama2-7B and Llama3. 1-
8B, widely used open backbones for downstream fine-tuning and dialogue (Touvron al. 2023bb
MetaA] (024).. We also consider domain/community models: Galactica for science-centric pretrain- ing (Taylor al. 2022). BLOOM as a large multilingual open model (Workshop) 2022), and Pythia for scaling/anal anallsis studies (Biderman et al. (023). Finally, we include popular instruction-tumm chat models derived from LLaMA/Llama (Vicuna-vl.5 (Zheng al., (023). Alpaca-7B (Taori
et al. 2023). Baize-7B (Xu et al. 2023)) well as Qwen3-8B as recent multilingual/resoning
baseline Yang et al. (2025).
Molecular Language Models. For molecular-domain LLMs spanning datasets, graph/3D align- ment, and reasoning, Mol-Instructiios (Fang et al., 2023) provides a large instruction-response corpus tailored to molecules, while trained Llama-based Mol-Instructions model themselves. Mol- Llama (Kim et al. 2025) targets general molecular understanding via multi--odal instruction tun- ing. 3D-MoLM (Li et al. 2024) equips an LM with 3D molecular encoder through dedi- cated projector for 3D-aware captioning and QA. LLaMo (Park et al.) (2024) integrates a molecular graph encoder with multi-level graph projector to bridge graphs and language. BioMedGPT-
28

---

Published as a conference paper at ICLR 2026
Table 18: Hyperparameter settings for benchmarks. For all the evaluated models and datasets, identical finetuning or inference parameters are applied for fair comparison.
Parameter
MoleculeQA
Mol-Instructions
PAMPA/BBBP
Batch size
2
32 BF16-Mixed
Precision
BF16-Mixed
BF16-Mixed
Max epochs
5
10
Accumulate grad batches
64
32
Weight decay
0.05
0.05
Initial LR
1. 1.00e-04
1.00e-04
Minimum LR
1.00c-05
1.00c-05
Warmup LR
1.00c-06
1.00c-06
Warmup steps
100
100
Scheduler
Cosine LR
Cosine LR
Temperature
0.1
0.1
0.1
Max input tokens
1024
1024
512
LM (Luo et al., 2023b) is a biomedical generative LM adapted for molecule- and biology-centric tasks. Hol-Reasoner (Zhao et al., 2025) emphasizes interpretable chemical reasoning with a two- stage procedure (supervised reasoning initialization followed by reinforcement learning). We fur- ther compare with graph-text align model HIGHT (Chen et al. 2024) on three tasks from the Mol-Instructions (Fang al. 2023 benchmark.
Benchmarks span factual QA, molecular captioning, property prediction, forward reaction predic- tion, retrosynthesis, reagent prediction, and ADME tasks, providing complementary coverage of accuracy, reasoning, and domain grounding. Baselines include both strong general LLMs and chemistry-specialized LLMs isolate EDT-Former 's contribution beyond backbone capacity.
BASELINE RESULTS
The official benchmark protocols and data splits adopted for all evaluations. For baselines whose reported performance trails ours by more than 10%, we cite the results directly from the benchmark sources (Lu et al. 2024 Fang al. 2023 Kim et al. 2025). For competitive models (within this margin) and for baselines not covered by the benchmark, we re-run both the baselines and our method under a unified, fair protocol using released code and configurations. Full implementation and evaluation details are provided in App. D.3
D.2 DATA PROCESSING
For structural inputs, we adopt Uni-Mol (Zhou et al. 2023) and MoleculeSTM (Liu et al. 2023). For Mol-LLaMA-Instraccs (Kim et al. 2025). we use the provided 3D coordinates for both pretraining and finetuning. For other datasets (e.g., Mol-Instructions (Fang et al. 2023)), we generate eleven conformers with RDKit and perform geometry optimization using MMFF (Merek Molecular Force Field Halgren (1996)), a classical small molecule force field that assigns bonded and nonbonded terms to atoms and bonds and minimizes molecular energy locally stable 3D) geometry. After optimization, RDKit is used to construct the 2D molecular graph (including atom and bond types, charges, hy hybridization, aromaticity, and stereochemical flags). Molecules that fail RDKit sanitiza- tion are filtered out. All processed data and scripts are released for reproducibility.
D.3 EVALUATION SETTINGS
As summarized in Tab. 18 all models evaluated on MoleculeQA (Lu et al. 2024 and Mol- Instructions (Fang al. fair comparison. For PAMPA and BBBP (Huang et al. 2021). I) all models are evaluated zero-shot with shared prompt template- BapplatteBBBPPPP. shown in Fig. and PAMPA following Mol-Llam (Kim) jet al.] 2025). For the 10-shot setting on MoleculeQA. the support set consists of 10 randomly se- lected examples fixed once and reused for every 10-shot model control variance.
2023) are finetuned under identical hyperparameter settings to ensure
29

---

Published as a conference paper at ICLR 2026
BBBP: Blood-Brate Baadd Barrier Penetration
Direct
System: "You are a drug discovery assistant tasked with predicting whether a molecule can penetrate blood-brain barrier (BBBP), Output the final decision strictly using the provided answer format string, in Your final answer should be formatted as cither: 'Final answer: Penetrant or 'Final answer: Non-penctrant"
User: "Determine the BBBP class of the given molccule. molccule..nMolocuuee MoMooccle <mol>."
Reasoning
System: "You are drug discovery assistant tasked with predicting whether molecule penetrate the blood- brain barrier (BBBP). Output the final decision strictly using the provided answer format string. Your final should be formatted either: 'Final answer: Penetrant' 'Final answer: Non-penetrant"
User: "Determine the BBBP class the given molecule and briefly explain your rationale, 'nMolccule <mol>."
Rich Instructions
System: "You are drug discovery assistant for BBBP prediction. Molecules that are lipophilic, appropriately sized, and not highly polar are more likely to penetrate the BBB, Consider: lipophilicity (logP), polar surface area, hydrogen bonding capacity, ionization state, and presence of efflux transporter motifs. Output the final decision strictly using the provided answer format string.'m Your final answer should be formatted as either: 'Final answer: Penetrant' 'Final answer: Non-penetrant"
User: " Determine BRBP class of given
<mol>."
Figure 9: Zero-shot prompt templates for BBBP GlooddBBrain Barrier Penetration). Three set- tings are evaluated: (i) Direct, which asks for binary decision; (ii) Reasoning, which requests a brief rationale before the decision; and (iii) Rich Instructions, which provides domain context (e.g., lipophilicity, PSA. H-bonding, ionization, efflux motifs) prior answering. All prompts enforce an identical output format: Final answer: Penetrant or Final answer: Non-penetrant.
Evaluation Metrics. (1) MoleculeQA (Lu et al. 2024): We report task-wise accuracy (fraction of correctly answered questions within each task). Average aceuracy the unweighted mean of the four task accuracies, and total accuracy is the overall correct/total across all questions pooled from the four tasks (count-based). Higher is better. (2) Hol-Instructions (Fang et al. 2023): For property prediction,
Mean Absolute Error (MAE), MAE E Uh Yi (lower is better). For molecular description generation, we report: BLEU-2/4 (n-gram precision with a brevity penalty; BLBU-2 uses up to bigrams, BLEU-4 up 4-grams: higher better), ROUGE-17 1/2 (recall-oriented overlap of unigrams/bigrans. higher is better), ROUGE-L (recall-oriented longest common sub- sequence; higher is better), and METEOR (harmonic mean of unigram precision and recall with stemming/synonnm matching and fragmentation penalty; higher is better). For SMILES/text gen- eration tasks, we additionally report Exact (fraction of predictions that exactly match the reference sequence; higher is better), Levenshtein (mean Levenshtein edit distance between predicted and ref- erence SMILES; lower better), RDKK/MACCS/MMraan FTS (mean fingerprint Tanimoto similarity computed with RDKit, MACCS, and Morgan fingerprints respectively; higher is better), Validity (fraction of generated SMILES that can be parsed and sanitized into valid molecules by RDKit; higher is better), and BERTScore contextual-embeddiin based semantic similarity between gener- ated and reference texts; higher is better).
D.4 EVALUATION ON MACROCYCLE DATASET-NPMMPD
Macrocycles represent challenging regime for molecular-language alignment due to their large ring systems, long-range dependencies, and highly flexible conformational space. To assess the ro- bustness of EDT-Formmr under this more demanding structural distribution, we additionally evaluate on the NPMMPD macrocycle benchmark (Feng et al. 2025). The dataset provides experimentally derived permeability measurements, and following prior work, focus the PAMPA Log Papp prediction task, which contains 326 unique macrocyclic molecules. This setting remains strictly zero-shot.
30

---

Published as a conference paper at ICLR 2026
Table 19: Zero-shot accuracy (%) on the NPMMPD PAMPA dataset under 13 prompt settings. The best (pink) and second-best (lightpink) results are highlighted.
Model
DI
D2
D5
D4
R1
R-2
R-3
RI-1
RI-2
RI-3
Bi
List
Conf
Avg.
General LIMSs
IJama2
89.84
52.56
48
41.37
52.49
48.03
41.08
57.14
52.67
49.78
41.3
42.24
41.3
48. 25
Llama3.1
46.54
42.51
38.68
36,10 10
47.33
44.25
37.68
51,4
46.69
43.82
41.3
41.61
43,48
43.18
Motecular LLMs
Mol-Ins.-I -lsmm22
41.94
30.98
37.30
24.32
44.63
38.23
41.00
44.55
42.02
38.68
45 34
40.37
40.37
39.21
Mol-Ins-LLamms.1
41.67
35.27
42.03
39.23
41.13
43.54
39.53
38.87
35.36
35.71
42.55
41.6
59.57
41.24
Mol-LLaMAA-
44.88
35.92
41.58 58
32,40
36.21
25.55 58
31.20
38.81
33.58
30.19
54.04
41.56
40.94
37.45
Mol-LLaMA3.1
39.79
30.86
36.46
23.63
42.93
42.93
42.95
38.11
35.14
27.71
49.88 09.84
42.95
44.72
38.31
Our
EDT-Former
62.66
53.93
58.39
55.9
45.65
55.59
51.81
Table 20: Zero-shot performance per prompt on the AMES toxicity prediction task from the TDC benchmark. We report accuracy (%) under 13 prompt settings. The best (pink) and second-bes (lightpink) results are highlighted.
Model
Size
D1
D2
R.1
R-2
R-3
RI-1
R1-2
RI-3
List
Conf
General
GPT-40
46.26
$4.95
55.43
55.00
47.11
55, 35.38
55.27
53.07
$3.90
55.72
55.17
55.48
$3.93
$3.59
Llama2
7B
45.74
54.92
54.81
54,81
45.80
54.81
54.81
52.34
53.44
54.81
54.88
54.13
54.68
53.08
LJama3.1
8B
46.13
$4.33
55,40
54.54
47.77
55.30
55.09
53.15
53.71
55 MH
54.81
56.19
52.54
53.46
Motecular LLMy
3D-MoLM
7B
51.75
54.43
52.95
50.29
52.35
53.46
53.90
52.75
53.59
55.48
53.03
50.28
51.91
53.01
LLaMo
49.50
52.31
49.80
50.70
50.34
53.11
54.89
54.93
51.67
Mol-Ima-U-lamaZ
8B
53.37
55.91
52.00
49.59
54.26
55.57
56.26
54.51
53.44
55.43
$1.86
$1.79
54.26
53.72
-llmmas
8B
44.98
48.07
53.71
50.48
44.70
45.19
44.91
45.53
52.13
53.71
54.63
43.81
54.95
48.98
Mal--LaMA-2
7.28
46.34
53:27
54.62
53.01
51.37
55.83
54.66
53.28
56.49
53.16
48.
52.29
55.51
52.97
Mol-LLaMAS.1
8.3B
53.51
$4.92
55.16
55.60
53.39
$1.68
51.21
53.78
55.36
53.05
51,12
51.81
55.96
54,50
Ours
EDT-Forrme
8.3B
55.53
52.69
56.69
58.79
58.59
Table 21: Zero-shot performance per prompt on the BACE classification task from the MoleculeNet benchmark. We report accuracy (%) under 13 prompt settings. The best (pink) and second- (lightpink) results are highlighted.
Model
D1
D2
D3
R-2
R-3
R1-2
R1-3
List
Conf
General LLMs
GPT-4o
40.72
37.22
38 88
30.46
40.14
39.30
40. 39
38 96
38. 16
39.
39.12
42.25
39.13
39 48
Llama2
7B
40.13
36,84
38.82
38.82
39.47
38.82
39.47
38.00
37.50
38.82
38.82
42.11
38.82
38.96
Lhmm3.1
40.13
38.16
38.82
43.42
38.82
41.45
38.82
40.79
38.10
46.71
39.47
38.16
42.76
40.44
Motecular LLASs
MoLM
39.74
39.53
48.17
50.44
37.91
35.14
36.58
37.29
45.79
42.12
45.73
36.62
43.52
41.43
39.08
48.06
36.37
37.73
41.65
47.77
37.39
45.37
41.91
Mol-Ens. -lamm2
SB
38.82
40.13
49.34
51.97
38.82
38.16
38.82
38.82
45.39
40.79
48.03
39.47
45.39
42.61
Mol-Ins.-Llamas.
40.22
36.36
39.77
36.76
38.37
39.68
67
39.76
36.11
46.91
40.12
Mol-LLaMA-2
7.28
44.49
48.68
40.67
52.63
38.82
30.07
38.82
38.03
38.57
38.46
44.74
38.16
38.82
41.54
Mol-ILaMAS.I
8:3B
37.50
39.29
38.
39.86
39.47
40.55
36.67
38.03
35.51
38.81
40140
37.14
41.0 .5
38.68
EDT-
8.3
16.05
55.92
37.50
52.98
46.71
61.18
60.53
55.26
44.08
44.74
38.82
53,29
11.45
49.12
For fair comparison across models with different pretraining protocols, we adopt the standardized 13 prompt configurations used in the main paper (Direct, Reasoning, Rich Instructions, and ten GPT-generated paraphrased templates). All models receive exactly the same molecule description and prompt wording under each configuration.
Table 19 reports accuracy under each prompt and the averaged accuracy. Across nearly all prompt variants, EDT-Forme achieves the strongest performance, demonstrating stable behavior even when prompts vary substantially in structure and reasoning depth. This robustness highlights the advan- tage of combining entropy-guided dynamic segmentation with anchor- alignment, especially for large and structurally complex macrocyclic systems where fixed-length token connectors typi- cally struggle.
D.5 DETAILED RESULTS OF PROPERTY PREDICTION TASKS PER PROMPT
Large language models exhibit substantial variability under different natural language instructions, especially for molecular property prediction. To systematically evaluate this prompt sensitivity,
31

---

Published as a conference paper at ICLR 2026
Table 22: Zero-shot performance per prompt on the BBBP classification task from the TDC bench- mark. We report accuracy (%) under 13 prompt settings. The best (pink) and second- best (lightpink) results are highlighted.
Model
Size
D1
D2
D3
D4
R1
R-2
R-3
RI-1
R1-2
R1-3
List
Conf
Avg.
Generai LLMs
GPT-4a
60.82
61.46
61.46
62.34
61.34
61.86
62.66
64.43
65. 10
65.11
65.91
66.37
66.89
63.52
Lhama
7B
37,57
38.33
38.50
39.22
$1.56
52.35
52.53
53.09
53.34
51.07
55.01
55.71
56.71
19.06
Llama3.1 1
SB
57.07
57.15
57.19
58.02
51,03
51.44
52.34
55.15
55.68
56.28
57.06
57.21
57.66
55.64
Motecular LLMs
3D-Mol.
7B
49.14
50.01
50.74 74
52.16
51.07
51.91
50:74
LLaMo
7B
55.44
55.81
56.42
56.85
55.15
55.16
56.16
56.91
58.67
57.52
59.31
59.29
60.08
57.18
Mol-Ima-LJaamm
8B
53.44
54.37
55,09
55.74
55,31
55.38
56.76
51.91
51.38
52.27
19.38
19.15
16.62
53.29
Mol-LLaMA-22
7.2B
53.57
55,
53.97
54,19
52.58
53.65
55.14
52.58
52.43
53.89
55.84
52.56
56.38
53.99
8.3B
59.54
59.54
55.56
53.18
59.08
54.67
53.29
58.92
57.45
Ours
EDT--Former
8.3
74.44
71.83
73,46
70.92
74,69
70.15
68.73
75,86
72.38
70.95
73.12
71.47
74.28
72.48
we design unified set of 13 prompts spanning three major categories (Direct, Reasoning, and Rich Instructions) together with three complementary variants that stress different aspects of LLM behavior. A full prompt example of BBBP is shown in Fig. I0
Direct prompts. These prompts provide a minimalistic instruction: the model asked to output only the final class label without any auxiliary context or justification. They represent the "pure clas- sification" setting and test whether the model can reliably make decision without verbal reasoning or domain cues.
Reasoning prompts. These prompts require the model to briefly explain the structural or physic- ochemical rationale (e.g., substructures, polarity, lipophílicity) before giving the final answer. Al- though the reasoning text is not evaluated, this setup probes whether intermediate verbalization stabilizes predictions helps models avoid superficial correlations.
Rich-instruction prompts. These prompts include additional domain background, such as AD- MET principles, known structure propeety heuristics, or high-level hints about BBBP, toxicity, or activity patterns. They simulate a more instructive "expert assistant" setting and evaluate whether models can utilize domain knowledge to improve prediction robustness.
Additional control-style prompts.
Beyond these three primary categories, include:
Binary: forces the LLM to behave like a strict binary classifier.
Confidence: encourages internal reflection before giving the final label.
Checklist: requires the LLM to follow a structured analysis procedure (identify substructures, consider polarity, check alerts, then decide).
These variants test different behavioral biases of LLMs and reveal how formatting, reasoning style, and instruction complexity influence prediction outputs.
Why prompt diversity matters. Property prediction tasks, especially classification benchmarks such BBBP, AMES, BACE, and toxicity tasks, are highly sensitive instruction phrasing. For tasks with balanced classes and clear structural cues (e.g., BBBP, AMES), most models remain relatively stable across the 13 prompts. However, for datasets with severe imbalance or safety- triggering semantics (e.g.. Clin'Tox, where the term "toxicity" may cause LLMs to refuse or hedge answers), prompt phrasing can cause large swings in accuracy. Therefore, evaluating with a single prompt can give misleading view of model's 's real capability.
Results across the 13 prompts. Tables in this section report complete accuracy across all prompts for all ten datasets (AMES-Tan 20 BACE-Tab. 21 BBBP-Tab Tab. 22 Tlin 23 DILI-Tab. 24 29). Across these 13 vari- 9 out of the 10 tasks, demonstrating
hERG- Tab. 25 Hab.Tab, 26 Tab. 27 Tab. 28 PAMPA-Tab Tab.
ants, EDT-Formmr achieves the best mean accuracy strong robustness to prompt variation. For most datasets, EDT-Former also ranks top-1 under the majority of individual prompts, indicating that its advantage is not tied to single favorable
32

---

Published as a conference paper at ICLR 2026
Direct Prompts
D-I: You are drug discavery assistant tasked with predicting whether a molecule can penctrate the blood-brain barrier (BBBP). Output the final decision stricily using the provided answer format string nnor final answer should be formatted as cither. FFinal answer: Penctrant' Final unswer: Non- penetrant". Deternine the BDBP class of the given melecule in Molecule <mol>"" "
molecule, you must decide whether it belongs to the positive (Active) or negative (Inactive) class defined by this dataser. Always follow the required answer format exnctly-lnYoo final answer should be formatted as either: Final answer. Active' of Final answer: lnactive', Classify the following molocule as Active or Inactive according to this datasct's task. 'MMolcuule "mol>."
D.2: "You are an expert molecular property prediction assistunt for binary task.
each molecule, you must decide whether it belongs the positive (Active) or negative (inuctive) class defined by this dataset. Always follow the required answer format exactly.'n 'nour final anewer should formatted as either: "Final answer: Active'
D-3: "You are an expect molecular property prediction assistant for binary
answer: Inactive'. Based structure, decide whether molecule should be labeled Active or Inactive for this task, Provide only the classification in the required final answer format Molocule <mol>."
D-4: "You are an expert molccular property prediction assistant for binary task. each molecule, you most decide whether it belongs the positive (Active) or negative (Inactive) class defined by this dataset. Always follow the required answer format exactly.'n 'nYour final answer should be formatted either: Final answer: Active' or Final answer: Inactive'. For molecule below, determine correct binary label (Active or Inactive) for this dataset and follow the exact final answer formar Molecule <mols."
Reasoning Prompts
R-I: "You are a drug discovery assistant tasked with predicting whether a molecule can penetrate the blood- brain barrier (8SBP), Output the final decision strietly using provided answer format string, 'oorr final answer should be formatted as cither. 'Final answer: Pouctrant or Final answer: Now- pee-penetrant. Determine the BBBP class of the given melocule and briefly explain your rationale in Molecale "mol>."
R-2: "You are expert molecular property prediction assistant. You should briefly reason about structural and physicochemical features of the molecule before deciding whether Active or Inactive. However, your final line must ohey the required unswer format Your final Active' 'Final answer: Inactive'. Briefly explain the main structura physicochonical reasons for your decision, thon give the final Active or lnactive label in the required format, format'n inMolcule <mel>>"
should formatted either. 'Final
R-3: "You are expert molccular property prodiction assistant. You should briefly resson about structural and physicochemical features of the molecule before deciding whether Active or Inactive. However, your final line must obey the required answer formatt'n Your final answer should be formatted either: Tinal answer. Active' Final answer: Inactive'. Consider potential substmuctures, electronic features, and global properties of the malecule, summarize your reasoning in one or two sentences, and then output the final Active or Innctive label using required final format MMmoauue <mol>."
Taf- Give Prompts
T-E: "You classification.. Use your knowledge of medicinal chemistry. ADMET principles, and structure' Sprogeey relationships to map molecules to the Active Inactive class. Do change the required Active' or 'Final answer: Inactive'. Using your understanding of typical structure'u200 3prrrrre relationships in drig discovery, decide whether the molccule should classified Active or Inactive
melccular property prediction assistant working specific binary dataset task (e.g.. activity, toxicity, or ADME
format 'our final answer should he formatted either: 'Final answer:
datasct, ioMoccule smol>." "
1:2: "You are molccular property prediction assistant working on a specific binary dataset task (e.g. activity, toxicity, or ADME classification). Use your knowledge of medicinal chemistry. ADMET principles, and structure 3prpoorrrry relationshhips to map molecules to the Active or Inactive class. Do not change the required answer format, Your final answer should be formatted as either: Tinal answer: Active' or FFiaal answer. Inactive'. Treat this realistic drig. discovery task. Infer whether the molecule belongs to the positive (Active) negative (Inactive) class defined by the datasct, and culput only the final label using the required format, Moooocale smol>."
I-3: "You are drug discovery assistant for BBBP prediction. Molccules that are lipophilic, appropriately sized, and not highly polar are more likely to penetrate the BBB. Consider: lipophilicitt (logP). polar surface area. hydrogen bonding capucity, ionization state, and presence of efflux transporter motifs. Output the final decision strictly using the provided answer format string 'Youu final formatted as either: 'Final answer: Penetrant' or 'Firal answer: Non-penetran". Determine the BBBP class of the given molecule 'nMoccuuee <mol>>" *
should be
Other Prompts
Binary: "You are a strict binary classifier for molecular property prediction. You are only allowed to output one of two labels, using the required format 'nour final answer must be exactly oue of: FFinal answar: Active' Final answer: Inactive'. Decide whether the molccule is Active or Inactive for this dataset. Do not output any explamation, only the final answer line. Moolecule <mol>."
Confidence: "You
careful molecular property prediction assistant. First internally estimate how confident you that the molecule is Active or Inactive, but do NOT include numerical prohabilities in the output After reasoning, output only the final decision using the required format. final answer should he formatied either: 'Final answer: Active' 'Final anewer: Inactive', Carefullly consider ambiguous or borderline cascs interally, but in your visiblc output provide only the final Active or Inactive label using the required final answer format ioMolcculo <mol>."
Checklist: "You are molecular property prediction assistant that follows a short checklist before answering: (1) identify key substructures; (2) consider lipophilicity, polarity, and size; (3) consider potential functional- group alerts; then (4) decide whether the molecule is Active Inacrive. Despite using this checklist, you must keep the final answer format unchanged Your final answer shoold be formatted as either: 'Final answer: Active' 'Final answer: Inactive'. Follow your internal cheoklist analyze the molccule and decide whether it should labeled Active or Inactive. In the visible output. you may include brief rationale. but the last line must follow the required final answer format inMoleuue <mols."
Figure 10: Example (BBBP task) of 13 prompt settings for the property prediction tasks from MoleculeNet and TDC benchmark. 10 Prompts are divided into three main categories- direct, rea- soning, and rich-instruction (task-info). Three more prompts are named binary, confidence, and checklist.
template. On particularly challenging toxicity tasks such as Clin all LLMs exhibit substantial prompt-to-promp variation due to class imbalance and occasional refusal-to-answer behavior, yet EDTT-Former still maintains consistently strong relative performance across prompts.
33

---

Published as a conference paper at ICLR 2026
Table 23: shot performance per prompt on the CLINTOX classification task from the Molecu- leNet benchmark. We report accuracy (%) under 13 prompt settings. The best (pink) and second- best (lightpink) results are highlighted.
Model
Size
D1
D2
D3
D44
R11
R-2
R-3
RI-1
R1-2
R1-3
List
Conf
Avg.
Generai LLMs
GPT-4a
93.01
10.50
5.81
8.69
82.46
9.82
10.02
71.10
10.98
7.20
5.91
18.08
8.87
26.
Lhama
7B
91.03
11.72
7.59
7.59
83:45
8.97
7.59
71.72
9.66
7.59
8.28
15.86
7.59
26.05
Llama3.1 1
8B
88.28
9.66
7.59
17.93
91.03
8.97
8.97
15.86
14.48
19.31
3.97
10.34
9.66
23.93
Motecular LLMs
3D-Mol.
7B
23.89
28.95
45.36
58.18
15.01
10.18
13.69
18.35
19.44
20:54
20.85
25.92
LLaMo
7B
23.81
31.04
47.08
56.79
15.56
9.98
11.81
18.84
17.01
21.34
10.02
19.52
19.30
25.55
Mol-Ima-LJaamm
30.34
46.90
15.17
8.97
13.79
17.93
21
Mol-Ins-Lhama3..
8B
47.25
18.37
14,92
11.58
43.88
8.97
8.61
56.70
10.53
9.90
17.17
17.54
53.26
26.79
Mol-LLaMA-22
7.2B
55.43
28.47
25. 00
48.97
$8.43
8.28
7.64
23.21
12.95
12.68
37.24
7.59
7.64
27.96
Mol-LLaMMA3.
8.3B
88.28
9.29
86.81
10.07
14.69
8.11
13.33
9.16
15.86
23.07
Ours
EDr--Former
8.3
75.86
48.28
30.34
$2.41
80.00
92.41
91,72
77.24
32.41
34.48
22.07
75.86
22.07
56.55
Table 24: Zero-shot performance per prompt on the DILI classification task from the TDC bench- mark. We report accuracy (%) under 13 prompt settings. The best and second-best are highlighted.
Model
Size
1>2
R-2
RE-1
R1-2
R1-3
List
Conf
Generoi LLMs
GPT-40
55.84
54.72
51,42
51.81
55.72
51.36
55,94
51.58
$7.40
53.17
51.86
50.11
51.12
53.47
IJamaZ
53.68
49.47
51.58
46.32
50.53
51.
Lhana3. 1
8B
49:17
48/42
51.74
52.63
50.53
51.58
52.63
50.53
50.53
60.00
51.58
51.74
52.63
52.31
Motecuiar LLMs
3D-MoLM
7B
50.52
48.00
43.26
41.16
54.85
47.28
50.94
50,19
56.65
51.81
45.02
53.08
41.34
48.78
I.aMo
7B
52.01
49.65
44.66
43.38
52.80
49.43
48.35
50.93
55.71
52.12
16
53.09
49.08
Mol-Ins. -laama2
8B
50.53
49.47
43.16
44,21
$1.58
.9.47
50.53
50.53
54.74
52.63
45.26
52.63
41.05
48.91
Mol-Ima-Llamm3..
47.37
55.79
49,
54.74
$2.63
45.26
52. 22.39
Mal-LLaMA--
7,2B
50.29
56.84
52.13
44.68
50.53
52.13
51.58
51.58
19.13
58.
52.63
53.19
51.58
51.92
Mol-LLaMA3.1
8.3
54.26
53.93
50,00
39
57.63
51,58
50.54
50,53
57,00
55. 68
53.76
51.65
48, 42
51.76
Owr
HDT-Farmer
833
60.61
52.86
52.31
50.00
50.70
40.00
35.59
51.06
49.33
45.61
49.28
50.27
Table 25: Zero-sho performance per prompt the HERG classification task from the TDC bench- mark. We report accuracy (%) under 13 prompt settings. The best and second-best results are highlighted.
Model
Size
D)1
122
R1
RI-1
RI-2
RI-3
Liss
Cont
Avg.
Generai LLMS
GPT-4u
21.97
73.65
73.43
73.39
31.10
72.50
74.51
55.75
69.17
71.17
70.63
68.57
73.39
61.01
Llama2
24.03
74.02
33.33
72.87
54.23
72.87
74.42
67.44
74.42
LJama3.1 1
8B
25.58
72.87
74.42
64. 34
25.58
75.97
75.19
56.59
69.77
70.54
75.19
72.87
76.74
64. (4.28
Moteeniar FIME
3D-MoLM
72.24
73.64
73.74
71,48
71.96
LLaMo
7B
71.15
72.91
60.78
51.80
58.25
72.55
76.12
27.67
64.45
61.75
55.7L
71.01
71.91
62.77
Mol-Ins, -Llama
8B
72.09
74.42
58.91
52.71
56.59
74.42
73.64
28.68
65.89
62.02
56.59
69.77
70.54
62.79
Mol-Ina. -Laamaa.
25.68
28.68
26.36
27.13
62.02
68.
51.94
Mol-I LMM-2
7.28
65.60
67. .44
68. 68.99
44.53
73.23
74.42
73.64
27.13
73.81
61.24
61.24
74.42
74.60
63.60
Mol-LLaMA3.1
8.3B
67.69
76.92
69.41
71.23
73.68
72.53
73.33
74.17
73.63
71.72
62.75
72.82
58.11
70.61
Ours
BIY--Former
8.38
65.60
75.19
74.42
74,92
72.58
73:23
75.00
74.42
74
74.39
74.47
72.22
74.42
73.46
D.6 EVALUATION ON MOLECULE DESIGN AND OPEN QUESTIONS FROM MOL-INSTRUCTIONS
Description-guided Molecule Design. The interiptionnguideer molecule design task evaluates a model's 's ability to generate valid molecular structures that follow language specifications. Following the official Mol-Instructions protocol, we strictly adopt the official splits and compare against all official baseline models under the unified benchmark setting.. All models are assessed using the official evaluation metrics: Exact Match, BLEU, Levenshtein distance, fingerprint-based similarity scores (RDK FTS, MACCS FTS, Morgan FTS), and chemical validity.
As shown in Table 30 EDT-Former achieves the strongest performance most metrics of the description-guie guidedd molecule design benchmark. In particular, it attains the best BLEU score, the lowest Levenshtein distance, and the highest RDK and MACCS fingerprint similarity scores, while also reaching perfect chemical validity ( (100% valid generations). Other strong molecular basclines such Mol15 and non-Instructions-Llnna models occasionally win individual metrics (e.g..
34

---

Published as a conference paper at ICLR 2026
Table 26: Per prompt zero-shot accuracy (%) on the HIA classification task from the TDC bench- mark, under 13 prompt settings, The best (pink) and second-best (lightpink) results are highlighted.
Model
D3
R-2
R-3
R1-2
R1-3
Conf
Avg.
General LLMs
GPT-4o
66.99
80,02
82
86.58
61.89
84.24
85.61
43.74
78 15
52.82
86.48
73.27
75.65
76.00
1Jama2
64.91
80.00
84.35
84.35
84.35
84.35
75.65
84.35
76.66
Lham3.11
SB
69.57
80.57
84.35
70.43
69.30
82.30
80.70
66.09
80.57
80.00
85.22
84.35
80.87
78.07
Molecular LLMs
3D-MoLM
83.18
76.07
81.66
LLaMo
7B
78.99
79.10
63. 13
44.68
75.29
82.03
81.49
63.66
67.14
73.81
73.36
83.29
79.73
72.75
Mol-mnssLLamma2
SB
78.26
79.13
62.66
46.96
73.91
81.74
83.48
64.35
66.09
73.04
73.04
82.61
80.87
72.78
Mol-Ins, Mol-Ins--Lham3...
15.22
16.09
63.48
41.74
51.30
15.65
15.65
30.13
77.39
75.65
77.19
10.87
81.74
50.95
Mol-LLaMA-22
7.28
48.45
68.70
65.79
33.91
62.64
84,07
82.61
58.29
8.1 82
72.81
54.78
84.35
94,21
67. 67.88
LaM33L
78.
78
Ours
EDT--aormer
8.3B
81.65
82.24
81.31
75.70
84.21
84,35
84,35
85.56
86.00
77.45
80.53
78.50
85.48
82.
Table 27: Zero-shot accuracy (%) under 13 prompt settings is reported on the HIV classification task from the MoleculeNet benchmark. The best and second-best results are highlighted.
Model
Size
D1
D2
D3
D4
R1
R-2
R-3
RI-1
R1-2
R1-3
Bi
List
Conf
Avg.
General LLMy
GPT44
9.20
3.10
6.35
25.53
50.18
Llama2
7B
11.85
8.13
3.60
3.75
4.06
4,90
4.63
14.72
21.22
23.31
22.13
25.22
50.14
15.20
Llama3.1
11.88
7.15
4.04
3.80
10.36
10.36
10.36
14.40
21.45
23.91
21.05
24.47
49.13
16.34
LEMss
3D M
78
8.54
19:36
41 64
611.2
7.98
11.96
8.09
13:42
26-96
28.67
59.62
14.98
5W.57
26 16
LLaMo
7B
7.28
16.39
42.14
60.21
6.26
8.55
8.83
13.17
23.08
26.55
36.27
12.11
56.29
24.39
3.92
15.33
59.60
3.84
5.56
6.73
12.76
23.78
23.31
34.41
11.98
57.09
22.82
Mol- Bns-Ins, Llama3.]
8B
10.84
14.64
12:16
11.67
7.93
9.11
9.62
5.12
9.69
12.86
37.51
15.12
60.23
16.70
Mol-LLaMA-2
4.45
36.31
22.51
Mol-LLaMA3.1
8.3B
6.51
7.91
5.33
31.89
5.40
15.63
8.12
14.83
23.84
23.48
43.41
25.87
72.28
21.89
Ours
EDT-Formmr
8.38
56.15
50.98
27.06
49.53
57.32
76.20
76.03
27.64
15.16
22.65
49.49
31,35
65.30
46.56
Table 28: zero-shot accuracy (%) under 13 prompt settings is reported on the PGP classification task from the TDC benchmark. The best and second-bes results are highlighted.
Model
Size
D1
D2
D3
D4
R-2
R-3
RI-1
RI-2
RI-3
List
Conf
Avg.
Generil LLMy
GPT-4o
49.85
48.67
$1.49
51.71
$1.05
49.53
53.48
43.42
52.44
49.16
49.59
51.54
50.79
50.21
Lluma2
7B
51.45
19.38
51.01
51.04
52.28
50.21
51.01
51.32
51.45
51.04
51.45
51,45
51.01
$1.09
Llunu3.1
48.71
51.87
52.28
53.11
43.98
51.87
51.45
30.26
$1.04
51.87
50.62
51,45
54.36
30.99
Motecular LEMSs
3D MoLM
19.09
52.54
19.13
49.82
54.62
53.88
56.35
18.62
80
49.79
45,21
53.83
51.17
51.55
49:29
51.63
52,20
54.09
47.47
51.89
51.00
51.22
Mol-Ins, -namma2
SB
$1.04
50.21
48.13
49.38
51.45
$1.87
$2.70
52.28
53.11
48.96
46.06
$2.70
$1.45
50.72
Mol-Ins--Llamkk.
57.26
51.87
53.94
57.26
48.96
50.21
$1.45
51.87
45.64
58.92
55.19
52,70
52.70
Mol-LLaMA-2
7,28
54.70
55.04
50.42
44.17
54.21
51.45
51.04
41.47
53.90 95
52.
53.53
51.25
51.25
51. 51.14
MoL-LLaMA3,1
8:3B
49.58
$1.09
53.07
50.00
$2.70
50,00
51.
51.67
52.28
49.06
46.84
51.97
49.79
50.73
EDT-Former
833
55.56
54.71
54.72
49.66
53,42
53.98
$9.31
56.65
51.55
57.14
45.05
59.17
59.44
54.64
exact match), but EDT-Former consistently ranks first or second across all metrics, indicating a better global trade-off between fidelity the textual description and chemical realism. As result, EDT-Former is not only effective on discriminative property prediction and QA, but also competitive with specialized molecular generators challenging, standardized molecule-design task.
Open Question and Answer. We further evaluate EDT-Former on the Open Question task from Mol- Instrutstucc,,,s, which measures free-form question answering over molecular contexts. Following the official benchmark protocol, all models are finetuned on the provided training split and evaluated on the held-out test set using BLEU, ROUGE-1, and BERTScore (Table General LLMs such as Alpaca, Baize, ChatGLM, LLaMA, Vicuna, and Galactica achieve only modest scores, while molec- ularly finetuned baselines
Mol-Ins. -Laaman2-- Mol-LLama) provide stronger but still uneven performance across metrics. In contrast, EDT-Former attains the best BLEU and BERTScore and competitive ROUGE-1, outperforming all Mol-Instructions baselines on overall natural-lange language quality. These results show that our connector based alignment not only preserves but even enhance the underfying L.MM's open-ended language capability on molecular QA.
35

---

Published as conference paper at ICLR 2026
Table 29: shot accuracy (%) under 13 prompt settings is reported on the PAMPA classification task from the TDC benchmark. The best and second-best results are highlighted.
Model
Avg.
General LLMs
GPT-4o
48.65
49.60
49.3
49.23
58.23
58.51
56.79
47.17
49.06
46.88
45.95
48.11
49.57
50.52
57.76
Lham3.
8B
$6.51
58.92
36.60
55.43
46.19
45.82
43.92
63.64
62.29
64.50
66.07
67.22
66.84
38.00
Molecular LLMss
3D-MoLM
62.00
LLaMo
7B
19.25
47.81
18:47
48.34
64.37
65.98
66.88
18.51
49.38
49.23
48.63
50.68
52.59
53.09
Mol-Ins.LLlmmz
49.63
49.94
50.09
47.74
31,16
33.60
32.35
38.15
40.63
39.58
39.21
39.98
38.22
40.78
Mol-Ins, Mol-Ins.LLlamA...
55.91
51.17
51.99
49.52
33.50
34,16
32.01
70:17
70.33
72.67
75.11
72.86
72.69
$7.39
Mol-LLaMA-2
7.28
75.68
77.85
50.20
79.80
79.61
81.17
79.75
67.90
09.14
67.54
68.30
69.36
68.67
74.23
LaM3LL
EDT-Formmr
8.3B
81.57
79.43
80.57
80.55
81.57
$4,05
85.37
83.78
82.26
81.32
81.95
83.36
84.65
82.34
Table 30: Results on the description- guided molecule design task from the Mol-Instructions bench- mark. Models finetuned and evaluated on the official splits. The best (pink) and second-best (lightpink) results are highlighted.
Model
Exactf
BLEUT
Levenshtein.
RDK FTST
MACC FTS1
Morgan FTS1
Validity
Alpaca
0.000
0.004
51.088
0.006
0.029
0.000
0.002
Baize
0.000
0.006
53.796
0.000
0.000
0.000
0.002
ChatGLM
0.000
0.004
53.157
0.005
0.000
0,000
0.005
LLaMa
0,000
0.003
59.864
0.005
0.000
0,000
0.003
Vicuna
0.000
0.006
60.356
0.006
0.001
0,000
0.001
Galactica
0.000
0.192
44.152
0.135
0.238
0.088
0.992
Text+Chem T5
0.097
0.508
41.819
0.352
0.474
0.353
0.721
MolT5
0.112
0.546
38.276
0.400
0.538
0.295
0.773
Mol-Ins.LLlama2
0.002
0.345
41.367
0.231
0.412
0.147
1.000
Mol-Ins.-Llama.
0.025
0.521
38.742
0.358
0.520
0.221
1.000
Mol-LLama
0.012
0.638
18.917
0.392
0.534
0.220
0.876
EDT--Former
0.016
0.652
16.826
0.401
0.560
0.251
1.000
Table 31: Results the Open Question task from the Mol-Instructions benchmark. Models finetuned and evaluated on the official splits. 3 metrics defined by the benchmark are reported. The best (pink) and second-best (lightpink) results are highlighted.
Model
BLEU
ROUGE-11
BertScoref
Alpaca
0.003
0.088
0.824
Baize
0.005
0.100
0.811
ChatGLM
0.003
0.090
0.795
LLaMa
0.003
0.100
0.814
Vicuna
0.004
0.097
0.814
Galactica
0.000
0.039
0.794
PMC.LLaME
0.007
0.788
0.625
Mol-Ins-L.Jama2
0.024
0.221
0.837
Mol--nns-Llama31
0.010
0.198
0.846
Mol-LLama
0.024
0.134
0.812
EDT-Former
0.101
0.286
0.857
D.7 NATURAL LANGUAGE ABILITY ANALYSIS
To assess whether alignment training affects the underlying natural language ability of the LLM backbone, we follow Mol-Instructions and evaluate the Open Question task zero-shot setting. This task requires free-form natural-language generation rather than molecular reasoning, making a sensitive probe for degradation caused by excessive domain-specifie fine-tuning. We compare performance before and after downstream alignment training molecular captioning and property prediction.
Mol-LLaMA (Kim et al. 2025) and EDT-Former trained under similar optimization settings (batch size 256, learning rate x 10 4 Adam optimizer, weight decay 0.05), but with a critical difference in training regime: Mol-LLaMA uses much heavier schedule (20 epochs), whereas
36

---

Published as a conference paper at ICLR 2026
Table 32: Performance on Open Question task from Mol-Instructions before and after finetuning us- ing their own finetuning settings on molecular captioning and property tasks. Mol-Llama (Llama3- based) and EDT-Former are reported.
Model
BLEU
ROUGE-IT It
BertScoref
Mol-Llama- Before
0.024
0.134
0.812
Mol-Llama-Afre
0.018
0.102
0.776
Average Drop
33.33%
31.37%
4.64%
EDT-Former Former-Before
0.024
0.137
0.819
0.024
0.137
0.819
Average Drop
0.00%
0.00%
0.00%
Table 33: Dataset overlap statistics molecular property prediction datasets from MoleculeNet and TDC benchmarks, comparing with the training data from Nol-Lllama-Instran
Dataset
Original
Removed
Cleaned
Overlap Rate
AMES
1454
305
1149
20.98%
BACE
152
1
151
0.66%
BBBP
406
112
294
27.59%
CLINTOX
145
32
113
22.07%
DILI
95
34
61
35.79%
HIV
4084
88
3996
2.15%
HERG
129
14
115
10.85%
HIA
22
19.13%
PAMPA
407
35
372
8.60%
PGP
241
40
201
16.60%
EDT-Former performs only 5 epochs. This contrast directly reflects each model's philosophy, which that Mol-LLaMA relies on extensive end-to-end fine-tuning, while EDT-Former is intentionally effective and preserves the natural language ability.
Tab. 32 the results. After fine-tuning, Mol-LLaMA suffers a substantial drop in natural- language quality, with BLEU decreasing by 33.33%, ROUGE-1 by 31.37%, and BERTScore by 4.64%. This confirms that heavy domain-specific training can erode the model's general language competence. In stark contrast, EDT-Foommr shows degradation across all metries: BLEU, ROUGE-1, and BERTScore remain unchanged before and after alignment training. In conclusion, EDTT-Former preserves full natural- natural-l fluency while still achieving strong molecular reason- ing and prediction performance.
D.8 DATA CONTAMINATION ANALYSIS
13-gram overlap (contamination) analysis. Following the GPT-3 practice (Brown et al.) 2020).
we measure data leakage by character-level n-gram overlap with n = 13. For text string let (I((x) be the multiset of all contiguous 13-grams. For a test item and a training document d, the overlap score is
overlap(1, G1a(t) n Gia(d) [9a(()]
(20)
and the contamination score for is max overlap(t,de d). We report the fraction of test items preset
whose contamination score exceeds a small threshold (e.g., any non-zero match Applying this procedure to our benchmarks, the test-train 13-gram overlap rate is below 5%. Given that we fine-tune only the connector (the LLM remains frozen) and for fewer than two epochs, memorization of instance-specio practice and does not affect our conclusions.
labels is unlikely; the observed overlap is within common LLM
Overlap Molecular Property Prediction Tasks. Beyond character-level 13-gram statistics, we further examine molecule-level overlap between the Mol-LLaMA- Instrrcct training set and our molecular property benchmarks. Using canonical SMILES to identify duplicates, compute, for each dataset, how many test molecules also appear in the training corpus. As summarized Table datasets ain Table several MoleculeNet/TIDC datasets exhibit non-trivial over-
37

---

Published as a conference paper at ICLR 2026
Table 34: Data overlap statistics between training data and five benchmark datasets from Mol- Instructions.
Dataset
Overlap Rate
Forward Reaction Prediction
0.21%
Reagent Prediction
0.69%
Retrosynthesis
1.20%
Property Prediction
0.36%
Molecular Captioning
1.93%
Table 35:
Performance comparison between the original and clean set of 10 molecular property pre- diction tasks from TDC and MoleculeNet. Average accuracy (%) of 13 prompt settings is reported.
BBBP BACE CLINTOX
HIV
PAMPA
DILI HERG HIA AMES PGP Average Avg. Drop
Original 72.48 49.12
56.55
46,56
82.34
50.27
73,46
82.15
55.20
54.64
66.34
Baseline +0.591%
Clean Set 73.17
51.59
53.48
46.66
82.93
50.08
71.64 82.41
56.17 57.19 66.74
Table 36: Accuracy (%) on tasks from MoleculeQA under official vs, scaffold splits.
Split
Structure
Source
Property
Application
Total
Acc
Official Split
74.55
72.39 72.18
50.71 50.87
48.58 42.85
68.34 68.65
Scaffold Split
76.28
+0.30%
lap (e.g.. DILI and BBBP with 35.79% and 27.59% shared molecules, respectively; AMES, CLIN- TOX, HIA, PAMPA, and PGP the 10-22% range), while others, such as BACE and HIV have relatively low overlap.
To directly test whether this overlap affects our conclusions, we construct clean test sets by re- moving all molecules whose canonical SMILES appear in Mol-LLaMA-Instrutt and re-evaluate EDT-Former on these pruned benchmarks. Table (35 reports the comparison between the original and clean evaluations. The per-dataset accuracies differ by at most few percentage points, and the overall average even slightly increases from 66.34 to 66.74 (a change of +0.59%). The rank- ing across tasks and our main observations remain unchanged. This robustness is consistent with our training protocol: the connector is trained on Mol-LLaMA-Instratt for only two epochs while the LLM backbone (excluding the embedding layer) is frozen, the model is encouraged to learn general structure structure-property patterns rather than memorize individual molecules or labels.
Overlap Mol-Instructions benchmarks. We also measure molecule-level overlap between Mol-LLaMA--nstrue and the five IostInnstustsion benchmarks used in our generative experiments. As shown in Table 34 these overlap rates are all very small. Given this already low level of dupli- cation, and the negligible effect of much higher overlap on the MoleculeNet/TDC property tasks, we do not re-run additional "clean-set" evaluations for Mol-Instructions. Taken together with the 13-gram analysis, these molecule-leve results indicate that potential data contamination is limited and does not materially impact the reported performance of EDT-Former.
Scaffold splits on MoleculeQA benchmark. To further rule out potential scaffold-level leak- age and to test generalization to novel chemotypes, we additionally construct scaffold split of MoleculeQA using Bemis-Murcko scaffolds, ensuring that no scaffold appears in both training and test sets. We keep the overall data size and task composition identical to the official split and re- train/evaluate EDT-Former under this more challenging setting. As shown in Table (36] the per-task accuracies on Structure, Source, Property, and Application remain highly similar, and the overall score even slightly improves (+0.30% total accuracy) compared the official split. This indicates that
generalizes well to unseen molecular scaffolds and that our MoleculeQA conclu- sions are not driven by scaffold overlap or memorization effects.
38

---

Published as a conference paper at ICLR 2026
Table 37: Average pairwise normalized mutual information (NMI) between fragmentation methods on the molecules from Mol-Llama- Instruct training set, reported as mean and median.
NMI (Mean)
NMI (Median)
Method
Entropy
BRICS
RECAP
Random
Entropy
BRICS
RECAP
Random
Entropy
1.000
0.484
0.401
0.177
1.000
0,547
0,505
0.176
BRICS
0.484
1.000
0.498
0.306
0,547
1.000
0,564
0.241
RECAP
0.401
0.498
1.000
0.498
0.505
0.564
1.000
0.564
Random
0.177
0.306
0.498
1.000
0.176
0.241
0.564
1.000
D.9 RATIONALE OF ENTROPY PATCHING
Our entropy-guidet patching is driven by lightweight Next-Atom Predictor (NAP), implemented as a small Transformer that models SMILES as sequence and outputs a distribution over the next token. For SMILES string I (.1..... IT), the NAP defines
pa( I<I),
He > Ea((v I<<e) Pog x<x),
where H2 is the token-level entropy (surprisal). We then place patch boundaries near local entropy peaks and group tokens between peaks into variable-length segments. The design goal is not to exactly recover chemically defined fragments, but to identify information-denss regions that are hard for Transformer to predict. These regions are expected carry rich structural context and thus are valuable for multimodal alignment with the LLM.
Chemical awareness of entropy-based patches. To verify that entropy-guided segmentation is still chemically meaningful, we compare it with two rule-based fragmentation schemes (BRICS (Jin- song al., (2044) and RECAP (Lewell et al. 1998)) as well as a random baseline (random shuffle of the split results from BRICS). For each molecule, we treat the fragment labels from method A and method B as two clusterings over atoms, z(4) and 2BB), and compute the normalized mutual information (NMI):
MI(z(A), (B,)
(T(2(1), (B))
Σ Piy log Pij 1
NM
P.Pj
where Ptj is the joint frequency of atoms assigned to fragment: i by A and fragment j by B, and (-) denotes the entropy of clustering. Tab. reports the pairwise NMI (mean and median across Mol-LLaMA-Inssieee
molecules). Entropy-basee patching shows substantial agreement with both BRICS and RECAP (c.g., mean NMI around 0.4-0.5), comparable to the BRICS-RECAP agree- ment, while all three methods are much less aligned with the random segmentation. This indicates that entropy peaks preferentially align with chemically meaningful breakpoints, even though they are derived purely from sequence prediction difficulty. In other words, our entropy-based patches are chemistry-awe aware enough to capture functional regions, yet remain tailored to what a Transformer- style sequence model finds informative.
Robustness across NAP model sizes. We also examine whether the entropy landscape is stable across different NAP capacities. Using the same NMI formulation, we compare patchings produced by three NAP variants (0.5M, 50M, and 500M parameters) and a random baseline. As shown in Table 38 the pairwise NMI between the three NAPs is very high (e.g., NMI > 0.85), while each has much lower agreement with random segmentation. This demonstrates that relative entropy patterns are largely invariant to NAP size: small NAP already identifies essentially the same entropy peaks and patches as much larger models. Consequently, we adopt the smallest variant in EDT-Former, which is computationally efficient yet yields stable, chemically aligned segments. This justifies our design choice of using lightweight NAP to drive -guided patching for multimodal alignment.
D.10 REPRODUCIBILITY
Our work is fully open source under the MIT license. Unlike many prior projects, we release end- to-end resources to reproduce every result in the paper: complete training and evaluation code (in-
39

---

Published as a conference paper at ICLR 2026
Table 38: Normalized Mutual Information (NMI) matrix of different sizes of the NAP model.
NAP variant
0.5M
50M
500M
Random
0.5M
0.94
0.86
0.12
0.94
0.92
0.13
500M
0.86
0.92
0.17
Random
0.12
0.86
0.17
1
Table 39: Extended evaluation results on Pampa dataset. Accuracy and FI scores are reported for each model across three prompt settings. Top and second-best results are highlighted.
Direct
Reasoning
RichInst.
Average
Models
Acc
Acc
F1
Acc
F1
Acc
GPT-4o
48.65
58.78
58.23
70.41
47.17
56.73
51.35
61.97
Mol-Inst-Llama2-78
49.63
63.19
31.16
39.84
38.18
46.61
39.66
49.88
Mol-LLaMA22-7
75.68
85.96
79.61
88.91
67.90
79.41
74.40
84.76
Llama3.1-8B
56.51
68.87
46.19
S7.22
63.64
75.88
55.45
67.32
Mol-Inst-Llamma.--88
55.91
69.84
33.50
39.84
70.47
81.46
$3.29
63.71
3D-MoLM-8B
46.93
50.00
62.43
64.86
76.45
53.93
65.56
LLaMo-SB
49.25
61.78
77.11
48.51
60.66
54.04
66.52
Mol-LLaMMA.L18.2B
63.55
75.32
76.72
72.48
83.51
66.80
78.52
EDYT-Former-8.38
81.57
89.92
81.57
89.74
83.78
91.15
82.31
90.27
cluding ablations), scripts for baselines and benchmarks, processed pretraining and finetuning datasets, and model weights. The repository is actively maintained and will serve the foun- dation for subsequent extensions. All materials are available via the anonymous GitHub link: https:// http://///onnmouss..mm 4open. www.:/cceeeneeeeeeee Bttp:////EDTFFTeeeeeeee
D.11 DATASET IMBALANCE ANALYSIS
To account for potential class imbalance in the Pampa dataset, we additionally report F1 scores alongside accuracy (shown in Tab. 39). As expected, EDT-Former achieves the best performance across both metrics under all prompting strategies, confirming that its gains are not driven by skewed label distributions but reflect genuine improvements molecular property prediction.
E EXTENDED ABLATIONS
Ablation studies model size, graph encoders, query token length, computational efficiency, and training hyperparameters are provided in this section. These analyses highlight how each factor influences performance and efficiency, underscoring the robustness of our approach.
E. IMPACT OF QUERY TOKEN LENGTH
As shown Tab. 40 increasing the number of anchor tokens improves accuracy, indicating that larger anchor set provides higher-bandwidthe more stable interface for aggregating global structural cues before conditioning the LLM. By contrast, expanding the maximum dynamic length from 64 to 128 yields a negligible change (e.g., 66.75 66.81 anchors) and even slight dip at anchors. This is expected: with a cap of 64, entropy-gun guided segmentation already covers the substructure count of most molecules, so raising the limit rarely increases the realized number of dynamic tokens and can introduce marginal redundancy. Overall, the best trade-off is achieved with 16 anchors and a max dynamic length of 64.
E.2 IMPACT OF MODEL SIZE
As shown in Tab. Qwen3 YYng et al. 2025) backbones below 2B parameters perform near random guessing on MoleculeQA, indicating insufficient capacity. Accuracy improves markedly scale reaches 4B, with further gains at 8B and 14B, albeit with diminishing returns from 8B ot 14B. use the Qwen3 family because it offers a broad spread of sizes for controlled comparison, and did evaluate larger backbones due to resource constraints.
40

---

Published as a conference paper at ICLR 2026
Table 40: Effect of anchor and dynamic to- ken length. Different settings evaluated the MolcueleQA dataset, with total accuracy re- ported for each ablation model. The top and second-best results are highlighted.
Table 41: Performance across different model sizes of backbone choices from Qwen3. Mod- els are evaluated on the MoleculeQA bench- mark with total accuracy reported. The top and second-best are highlighted.
Anchor Length
Max Dyn. Length
Accuracy
Model Size
Accuracy
4
64
64.29
0.6B
27.23
8
64
66.75
1.7B
26.52
128
66.81
36.82
16
68.34
8B
54.79
16
128
68.03
14B
56.84
Table 42: Ablation finetuning hyperparameters MoleculeQA benchmark. Accuracy is reported for different initial learning rates, global batch sizes, and training epochs.
Init LR
Accuracy.
Global Batchsize
Accuracy
Training Epochs
Accuracy
5.0 x 10
67.92
32
67.97
2
65.23
1.0 x 10-4
68.34
64
68.08
4
67.92
5.0 x 10-5
67.82
128
68.34
6
68.34
1.0 x 10-6
67.09
256
68.11
8
68.13
E.3 ANALYSIS OF HYPERPARAMETER
As summarized in Tab. varying the initial learning rate, global batch size, and training epochs produces only minor fluctuations in accuracy. This stability indicates that performance gains primar- ily arise from our multimodal fusion design and catropy-guider structural cues rather than aggressive hyperparameter tuning.
E.4 IMPACT GRAPH ENCODERS
We evaluated stronger molecular encoders (e.g.. Uni-Mol-v2 (Ji 2024) ) under matched settings and observed only minor accuracy gains relative to our base encoder. This suggests that, beyond reasonable encoder quality threshold, the primary bottleneck for molecular understanding lies not in the encoder itself but in the multimodal fusion interface to the I.I.M. In our framework, the entropy- guided substructure tokens and dynamic query-anchor fusion contribute the dominant improvements by exposing chemically salient evidence the LLM; upgrading the encoder yields diminishing returns compared to better fusion.
E.5 IMPACT OF DIFFERENT GLOBAL/LOCAL TOKEN BUDGETS
We first study how the connector's token budget should be allocated between global (anchor) tokens and local entropy-based dynamic tokens. Under a fixed total budget of 64 tokens, we train three variants: (i) Dynamic-only (0 anchors / 64 dynamic). (ii) Anchor-only (64 anchors / dynamic), and (iii) Hybrid (32 anchors / 32 dynamic). We then evaluate each variant on five molecular prop- erty tasks (BACE, BBBP, Clin'Tox, HIA, PAMPA) and report accuracies averaged over six shared prompts per task. As shown in Table 43 the hybrid configuration consistently achieves the best average performance, outperforming both dynamic-only and anchor-only designs. Among the single-source variants, the dynamic-only model clearly surpasses the anchor-only one, indicating that entropy-guides local patches more informative than global learned queries when used in isolation. Overall, these results support our design choice: combining a small set of global anchors with content-adaptive dynamic tokens yields the strongest and most robust connector.
41

---

Published as a conference paper at ICLR 2026
Table 43: Different token budget settings ablation on 5 molecular property tasks.
Anchor
Dynamic
BACE
BBBP
CLINTOX
HIA
PAMPA
Avg.
0
64
42.15
74.26
44.25
70.73
61.05
58.49
64
0
41.23
71.58
42.50
69.00
56.41
56.14
32
32
45.55
70.28
47.69
78.87
64.45
61.37
Table 44: Ablation of LLM finetuning strategies on four tasks from the MoleculeQA benchmark and the overall accuracy (%).
Setting
Structure
Source
Property
Application
Total
All LLM Parameters Frozen
74.55
72.39
50.71
48.58
68.34
Unfrozen Embedding Layer
74.46 76.25
72.19
50.30
48.61
68.44
Full LLM Finetuning
75.48
52.71
51.25
70.50
Table 45: Ablation over training corpora on four tasks from MoleculeQA and the overall accuracy.
Training Corpus
Dataset Size
Structure
Source
Property
Application
Total
Mol-L.lmma-Instenn
312M
74.55
72.39
50.71
48.58
68.34
PubChemQA
325M
73.61
74.81
50.39
48.61
68.36
3D-MolT
646M
72.27
76.85
52.26
49.94
68.48
E.6 IMPACT OF TRAINING OF LLM EMBEDDING LAYER
Our default setting follows common multimodal practice by freezing all transformer blocks of the LLM while allowing the token embedding layer to update. This is motivated by the need to as- sign meaningful representations to newly introduced molecule-related tokens (symbol tokens, spe- cial markers, etc.). To disentangle the effect of this design, we compare three configurations MoleculeQA: (i) All LLM parameters frozen, (ii) Unfrozen embedding layer only (our default), and (iii) Full LLM finetuning. Table 44 summarizes the results. As expected, fully finetuning the LLM yields the highest overall score, but at the cost of a much heavier and less modular training procedure that risks degrading general language ability. More interestingly, performance gap between "all froze" and "embedding-only" training is very small: unfreezing the embedding layer brings only marginal gains, while EDT--ormmr with completely frozen LLM remains strong. This indicates that the connector itself' is doing most of the alignment work: can effectively adapt molecular representations into the existing token space of the backbone with- out relying heavily on embedding updates. In practice, we keep the embedding layer trainable for flexibility, but these results show that even stricter freezing is viable.
observe that the
E.7 IMPACT OF DIFFERENT TRAINING CORPUS
Finally, investigate whether EDT-Former's performance is tied a specific instruction cor- pus. Using the same training recipe, we align the connector with three alternative datasets: Mol- LLaMA-Instruct, PubChemQA, and 3D-MolT, whose sizes range from 3300 650M in- couction--mlecuut pairs. We then evaluate the resulting models on the four MoleculeQA tasks. As shown in Table 455 all three corpora lead to very similar scores, with only minor fluctuations in per-task accuracy and total average. Notably, the larger 3D-MolT corpus provides only modest improvements over the smaller alternatives. This suggests that EDT-Former primarily requires a reasonable amount of heterogeneous molecule-text data to learn a good alignment; the exact choice among existing high-quality corpora is not the main driver of its advantage. In other words, gains do not rely specially curated training set, but stem from the entropy- -uiiy--ggieddd connector architecture itself.
E.8 INFERENCE EITERENCE-TIME ABLATION OF DYNAMIC TOKENS AND ANCHORS
To verify that both dynamic tokens and anchors are functionally used at test time, we perform inference-only ablations trained EDT-Former without modifying its training procedure. Specif- ically, we compare the full model (anchors dynamic tokens) with four variants: (i) No dynamic tokens, where all dynamic tokens removed and only anchors are kept; (ii) Random dynamic
42

---

Published as a conference paper at ICLR 2026
Table 46: Inference-time ablations of dynamic tokens and anchors. Accuracy (%) from prompts task is reported.
Variant
Pampa
ВBBP 1
Full EDT-Formee
82.34
72.48
No dynamic tokens
54.97
51.29
Random dynamic tokens
64.25
54.71
No anchors
79.26
66.38
Random anchors
80.10
66.74
Table 47: Functional-gros group hallucination 200 PubChem molecules (lower is better). Non-Rate percentage of non-meaningful outputs; Hallucination Rate: percentage of molecules where at least non-existing functional group is mentioned.
Model
Non-Rate |
Hallucination Rate |
GPT-4o
0.0
41.5
Llama2
0.0
50.5
Llama3.1 1
0.0
45,0
3D-MoLM
34.0
81.0
LLaMo
27.0
63.5
Mol-Inst.LLlama2
14.5
57.0
Mol-Inst--llama3.
14.0
54.5
Mol-I.aaMA22
7.5
36.5
Mo1-LLaMA-3.1
7.0
39.5
HIGHT
22.5
36.0
EDT-Former
3,5
19.5
kens, where dynamic tokens replaced by random vectors; (iii) No anchors, where only dynamic tokens are used; and (iv) Random anchors, where anchors are replaced by random vectors. As shown in Table 46 removing
randomizing either component leads clear accuracy drops PAMPA and BBBP, with the largest degradation observed when discarding dynamic tokens entirely. These results confirm that both anchors and dynamic tokens contribute non-trivially inference time, and that neither component is redundant.
E.9 HALLUCINATION ANALYSIS
Hallucination evaluation dataset. To investigate the hallucination of our model, construct a targeted benchmark for functional-group hallucination. Concretely, we randomly sample 200 molecules from PubChem and obtain their human-readaa label texts. For each molecule, use the GPT-5 API to summarize the label into a canonical list of functional group names, and cross- check these groups using RDKit-based (Landrum) 2013) functional group decomposition with man- ual verification. This yields, for each molecule, a vetted set of functional groups that we treat as the ground-truth reference for hallucination analysis.
Evaluation protocol. For every model, we prompt it generate a natural language description of the functional groups present in the given molecule. We then compare the model's output with the ground-truth functional groups by using the GPT-5 API to judge whether the model has hallucinated any functional group that does not exist in the molecule. We report two metrics: (i) Hallucination Rate, the percentage of molecules where least one non-existing functional group is mentioned; and (ii) Non-Rate, the percentage of molecules where the model fails to produce meaningful, readable answer (e.g., broken text or uninterpretable tokens). Explicit abstentions such as don't know" are not counted as hallucinations and are excluded from Non-Rate, as they are acceptable behavior for safety-criticn applications.
The quantitative results are summarized in Table 47 EDT-Former achieves the lowest hallucination rate among all methods, roughly halving hallucinations compared strong molecu- lar LLM baselines (c.g.. Mol-LLaMA and HIGHT), while also maintaining a low Non-Rate that indicates it usually answers in coherent natural language rather than failing silently. These results
Results.
43

---

Published as a conference paper at ICLR 2026
provide direct quantitative evidence that reducing structural information loss via entropy-guided dy- namic tokens and sequenc alignment not only improves downstream metrics, but also sub- stantially mitigates functional group hallucination, directly addressing motivation around "loss of structure."
F LIMITATIONS AND FUTURE WORK
F.1 LIMITATIONS
While EDT-Former demonstrates broad applicability across molecular tasks, we note several cur- rent constraints that highlight opportunities for improvement. As generalist molecular LLM, it does not yet consistently match highly specialized classifiers that are trained and tuned per dataset- pattern commonly observed in the Mol-LLM literature. In this study, also did not scale larger backbones due to practical compute limits. In addition, we did not incorporate synthetic "reasoning" corpora, as widely used resources are predominantly GPT-generated and lack reliable human- verified rationales. which could confound conclusions. Finally, the present system answers end-to-end without external tools, which can increase the chance of occasional hallucinations or urithmetic/chemistyy slips. We emphasize that these constraints are scope choices rather than fun- damental barriers and frame clear directions for progress.
F.2
FUTURE WORK
Building these findings, plan to close the generalist-specialis gap via lightweight task adapters and, where feasible, moderate scaling both backbone and bridge with efficient fine- tuning. We will curate and release verifi-e molecular reasoning benchmark replace purely synthetic rationales, and integrate tool-augmented agents (e.g., RDKit calculators (Landrum 2013), literature retrieval, unit/constraint checks) with self-verification reduce hallucinations and enforce domain validity. Finally, we will broaden evaluation to include robustness under distribution shift, calibration, and abstention, and cost/latency reporting, make EDT-Former both reliable and practical in real-world molecular workflows.
G
LLM USAGE
Large language models (LLMs) were used only for light editorial assistance (grammar wording polish and minor LaTeX phrasing). They did not contribute research ideation, experimental de- sign, data collection, analysis, result writing. Separately, LLMs appear in experiments solely as baselines/backbones within the method; all runs, configurations, and analyses were conducted and verified by the authors. The authors take full responsibility for the paper's 's contents.
44


Reviews:
[{"review_id": "N3GDvggGsY", "reviewer": "Reviewer_y4JF", "summary": "The paper introduces EDT-Former, for aligning molecular graphs with large language models (LLMs) under frozen-backbone settings. The model addresses two issues common in molecular graph–language alignment: (1) loss of structural fidelity caused by fixed-length, Q-Former-style connectors; and (2) inefficient fine-tuning due to large backbone updates. EDT-Former proposes two main mechanisms—Entropy-Guided Patching and a Dynamic Query Transformer—to generate substructure-aware dynamic tokens and integrate them with static anchors for efficient alignment. The corresponding EDT-Former shows promising results on the evaluated benchmarks, including MoleculeQA, Mol-Instructions (molecule captioning and property prediction), Pampa, and BBBP.", "questions": "- Regarding experimental setting of Figure 5. I'm curious about how the experiments in Figure 5 were conducted. Does it correspond to retraining the model excluding each component, or excluding each element only at inference time from the full model inference? There doesn't seem to be an explanation of the experimental setting for Figure 5.\n- On the property prediction benchmark choice. While EDT-Former's zero-shot performance in Table 2 is impressive, I wonder why only BBBP was used among the widely comparable evaluation benchmarks for property prediction such as MoleculeNet that are commonly used among molecular LLMs? Given that the evaluation is for zero-shot performance, using only BBBP and Pampa as property prediction benchmarks seems to both forfeit the advantages of models capable of zero-shot inference and provide a non-comprehensive evaluation. As the authors stated, since EDT-Former is not dependent on LLM finetuning, expanding the evaluation benchmarks would be a viable option as model retraining is not required for benchmark selection. Referring to widely used evaluation benchmarks such as MoleculeNet could easily demonstrate directly convincing results.\n- In cases where molecules contain highly repetitive or symmetric substructures (many molecules could have long carbon chain, resulting in CCCC…), how does entropy-guided segmentation handle indistinguishable regions? Is there a mechanism to prevent redundant substructure tokens?\n- The benchmarks used focus mainly on small molecules. Can the authors discuss or show any evidence regarding scalability to large macrocycles, where token budget and entropy dynamics might differ substantially?\n- In Figure 6, attention patterns are qualitatively aligned to high-entropy regions. Have the authors considered quantifying this alignment (e.g., attention–entropy correlation) to strengthen the link between entropy peaks and attention interpretability?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "- Good motivation\n    - This work is well-motivated by the need for a dynamic length graph token that captures molecular substructure information.\n- Methodological novelty\n    - The proposed tokenization is novel, based on entropy-guided segmentation for molecules based on uncertainty peaks from a next-atom predictor, offering a data-driven and deterministic patching mechanism. This design appears to be suitable for learning the representation of molecular functional groups, considering the dynamic size of the molecule.\n- Connector-only alignment design\n    - The Dynamic Query Transformer establishes a modular bridge between frozen molecular encoders and frozen LLMs, requiring no or minimal gradient updates to the LLM. This contributes to computational efficiency and strong performance scalability, aligning with the authors’ focus on low-cost, high-fidelity multimodal integration.", "weaknesses": "Overall, I find the proposed method interesting and reasonable (have a different opinion regarding NAP, though); my concerns primarily relate to the experimental setting and the demonstration of the author's hypothesis.\nI hope these are properly addressed in the rebuttal phase.\n\n- Ambiguity in the Description of Experimental Settings and Results\n    - (Major) In line 312, they mention evaluating with Direct, Reasoning, and Rich Instructions prompting to reduce prompt sensitivity, but there is no information in the main body about whether these 3 prompting strategies align with the instructions used for finetuning each baseline model. This is a factor that could largely vary each model's performance and is easy to miss without background knowledge of each model's experimental setting. This descriptive gap makes it difficult to understand if the experiments were conducted fairly.\n    - (Major) Mol-LLaMA is missing from the comparison baselines in Table 4, yet Mol-LLaMA also provides evaluation results for molecule captioning and property prediction in Mol-instructions (Table 17 from Mol-LLaMA), so including it seems appropriate. In line 371, the authors claim Q1, but Mol-LLaMA's experimental results appear to show higher performance, which needs to be addressed. The following is Table 17 from Mol-LLaMA. Please let me know if I have misunderstood anything.\n| Models                         | BLUE-2 | BLUE-4 | ROUGE-1 | ROUGE-2 | ROUGE-L | METEOR | MAE     |\n|--------------------------------|--------|--------|----------|----------|----------|---------|---------|\n| Mol-LLaMA (LLaMA-2)            | 0.478  | 0.425  | 0.761    | 0.698    | 0.750    | 0.701   | 0.0035  |\n| Mol-LLaMA (LLaMA-3)            | 0.476  | 0.426  | 0.767    | 0.708    | 0.759    | 0.707   | 0.0039  |\n\n    - In line 301, the authors state they used Mol-Instructions' evaluation benchmark; however, the provided experimental results used only 2 tasks (molecule captioning and property prediction) out of the total 17 tasks in Mol-Instructions. This could be misunderstood as results spanning the entire Mol-Instructions dataset.\n    - In many sections including Abstract line 20, Introduction lines 114, 119, etc., they describe LLMs as fully frozen, yet in C.3 line 1238, they state that the LLM embedding layer was tuned. Which explanation is correct?\n- (Major) Fairness of Experimental Settings\n    - Fair comparison of baseline models in zero-shot evaluation\n        - As mentioned above, when measuring the zero-shot performance of finetuned models in Table 2, prompting strategies that are not aligned with the finetuning dataset can severely impair model performance. This is because models finetuned on molecular tasks can easily lose natural language following ability, which can be immediately confirmed by running inference on molecular LLMs such as BioT5, LlaSMol, LLaMo, etc., using their HuggingFace open-sourced models. This could create an evaluation setting that is favorable to EDG-Former, which is the only model not finetuned LLM backbone (except for embedding layer). To easily address these concerns, it is suggested that the performance for each prompting strategy be reported before averaging.\n    - Data contamination\n        - The Mol-LLaMa-Instruct used by the authors for EDT-Former's alignment training contains GPT-4o generated molecule captions augmented from Pubchem324K, which can be used as a dataset for molecule captioning tasks. Considering that the source of the Mol-Instructions' molecule captioning dataset used in Table 4 experiments is also PubChem, there is a need to clearly describe whether data contamination exists between the train-test splits of these two datasets. Although the authors claim in D.4 that they performed character-level 13-gram analysis with reference to GPT-3, given that GPT-3 deals with general-purpose text data, not focusing on molecular tasks, this seems insufficient to verify data contamination in molecular tasks. In addition, instead of stating the exact figure for the ratio of data with overlapping 13-characters, the authors merely state it's below 5%, but a figure close to 5% could still be perceptible to humans. To clearly resolve these concerns, it is suggested that a contamination analysis based on scaffold split, which is widely chosen in the molecular domain, be provided.\n- Regarding demonstration of Q3\n    - (Major) Comprarision with fixed length token with enough molecule tokens\n        - The authors introduced dynamic molecule tokenization to address the problem of fixed length molecule tokens losing molecular features due to limited length. However, in E.1, they state through dynamic token maximum length ablation that 64 molecule tokens are sufficient. Regarding this, considering the sequence max length budget of current LLMs, using a fixed length token of 64 is not a significant burden. Is there a performance difference compared to using fixed length tokens of 64? To prove Q3, a comparison of performance between using fixed length tokens of sufficient size and EDT-Former is necessary, which I believe is an essential ablation study given the message of the paper.\n    - Ablation of dynamic token in inference time\n        - Since EDT-Former uses fixed length molecule tokens concatenated with dynamic tokens, there is a need to verify whether it might actually be bypassing dynamic tokens and only using fixed length tokens. This seems easy to check at the inference level - I'm curious about the performance when replacing dynamic tokens and fixed tokens with random or dummy tokens during inference.\n- (Major) Limited justification for entropy predictor selection\n    - The entropy estimation relies on a simplistic GPT-2–based next-atom predictor (NAP) trained on SMILES. While computationally light, the rationale for this choice is mostly under-investigated, given that next-atom in sequence does not necessarily align with molecular subgraph structure, as originally the author aimed to represent by EDT-Former.\n- Interpretive analysis remains descriptive\n    - While attention visualizations (Fig. 6) qualitatively support the claim that dynamic tokens attend to “structural transitions”, the study lacks quantitative or diagnostic analysis. Although I don’t think it is necessary to show that the hypothesis holds for almost all molecules, at least line 250 should be adjusted in light of current experimental evidence. In addition, analysis on specific failure modes (e.g., mis-segmentation, entropy misalignment, or redundant patches) is necessary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "t4DdNfZ20C", "reviewer": "Reviewer_x8D4", "summary": "This work studies the multimodal LLMs for aligning LLMs with molecular graphs. The authors propose  EDT-Former, a connector-only approach that generates fixed query tokens for molecules in different lengths. EDT-Former includes (i) Entropy-Guided Patching, which uses next-atom surprisal peaks from a lightweight SMILES predictor to segment molecules into substructure-aware patches, and (ii) a Dynamic Query Transformer that fuses these variable-length “dynamic tokens” with a small set of learnable modality anchors before projecting into the LLM space. Experiments across multiple tasks and benchmarks show the effectiveness of EDT-Former.", "questions": "Please find the details in the section above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work targets on fixed-token bottleneck in graph-LLM alignment, which is timely and critical;\n\n2. The proposed approach is novel and interesting;\n\n3. There are significant empirical improvements;", "weaknesses": "1. The benchmarked tasks seem to be limited. For example, can this approach be applied to other tasks in Mol-Instructions?\n\n2. The empirical comparison seems not to be fair, as EDT-Former uses different training corpus with other baseline approaches. Given the efficiency of the proposed approach, can EDT-Former be applied and ablated with different instruction training data?\n\n3. Lack of comparison and discussion with a closely related work [1]. For example, can the proposed tokenization scheme mitigate the hallucination issue mentioned in [1]?\n\n- HIGHT: Hierarchical Graph Tokenization for Graph-Language Alignment, ICML'25.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OPJxVFD0iR", "reviewer": "Reviewer_x1GF", "summary": "This paper addresses two major limitations of existing graph-Large Language Model (LLM) frameworks for molecular understanding: 1) information loss from using fixed-length token connectors (e.g., Q-Former), which compress complex, variable-sized molecular graphs into a static representation, and 2) the high computational cost and poor generalization resulting from fine-tuning the entire LLM backbone.\nThe paper proposes **EDT-Former**, an \"Entropy-guided Dynamic Token Transformer,\" as a novel connector. The key idea is to generate a *variable* number of tokens that are aligned with a molecule's structural complexity. This is achieved through a two-part mechanism: entropy-guided patching and a dynamic query transformer. The paper demonstrates the best performance on a wide range of benchmarks.", "questions": "Please refer to the weaknesses section.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to follow.\n- The paper proposes a novel query Transformer. Most works simply abstract molecules into queries, not considering the stereochemistry and structural context in the molecule. Another line of work exploits rule-based algorithms to extract the meaningful substructures. Different from them, the proposed work automatically extracts substructures by applying entropy-based patching segments.\n- Experimental results demonstrate the effectiveness of EDT-Former with its superior performance compared to molecular LLMs and General LLMs.\n- The paper provides a rigorous analysis, which helps in understanding the contribution of the proposed component.", "weaknesses": "- The entire entropy-patching mechanism is based on a 1D SMILES sequence. The properties of a SMILES string (and thus its entropy profile) can change based on the canonicalization algorithm used or if non-canonical strings are permitted. The paper does not discuss the robustness of the patching mechanism to different, yet chemically equivalent, SMILES representations of the same molecule.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "f06aL9FyIy", "reviewer": "Reviewer_XpYi", "summary": "This paper introduces ​​EDT-Former​​, a novel method for aligning molecular graphs with Large Language Models (LLMs). It addresses key limitations of existing approaches, such as Q-Former-style bridges, which use a fixed number of tokens that often lead to loss of stereochemical and substructural details, especially in larger molecules.\nThe core innovation lies in its two components: an ​​Entropy-Guided Patching​​ strategy that dynamically segments molecules into informative, variable-length tokens based on structural uncertainty, and a ​​Dynamic Query Transformer​​ that integrates these tokens with static modality anchors. EDT-Former achieves this alignment without fine-tuning the LLM backbone, enabling highly efficient training.", "questions": "Please see weaknesses", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper introduces an innovative solution to a clear limitation of existing Q-Former-style bridges.\n\n2. The results demonstrate state-of-the-art performance across multiple benchmarks.\n\n3. The framework is trained without fine-tuning the LLM backbone, which is efficient.", "weaknesses": "1. Evaluation is centered on question-answering and property prediction. The method's effectiveness on more challenging generative tasks, such as molecule generation,  remains an open and important question.\n\n2. This paper should discuss the difference with more molecular graph-text pretraining frameworks, such as [1,2]. \n\n3. This paper uses the SMILES string to represent the molecular structure, which might not capture the structure of the molecule well. The graph structure or 3D structure could contain more structural information.\n\n[1] Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing\n\n[2] Advancing Molecular Graph-Text Pre-training via Fine-grained Alignment", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)