Omni-Mol: Multitask Molecular Model for Any-to-any Modalities
Authors: Chengxin Hu
Hao Li
Yihe Yuan
Zezheng Song
Chenyang Zhao
Haixin Wang
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models
Molecular Science
Multi-modal Learning
Primary area: deep_learning
TL;DR: We present Omni-Mol, a multitask molecular model that can solve any-to-any modality tasks

Abstract: In the molecular domain, numerous studies have explored the use of multimodal large language models (LLMs) to construct a general-purpose, multi-task molecular model. However, these efforts are still far from achieving a truly universal molecular model. We identify three key challenges in this endeavor: (1) Existing molecular task datasets are typically small in scale and lack comprehensive domain coverage. (2) Tasks from different molecular subfields are difficult to effectively learn jointly through LLMs due to significant distributional shifts and competition among tasks, which introduces instability in the learning process. (3) Both inter-task and intra-task molecular representations demand different intrinsic dimensions in the language space, making it challenging to balance between redundancy and insufficiency in language model representations. To address these challenges, we innovatively categorize existing small-molecule tasks into four types: Mol2Mol, Mol2Text, Mol2Num, and Text2Mol. We then collect a dataset encompassing over 16 tasks with more than 1.4 million samples, making it the largest molecular instruction-tuning dataset to date. Leveraging the extensive pretraining of LLMs on existing chemical literature, we propose a novel multimodal LLM framework, named **Omni-Mol**, which unifies all small-molecule tasks and supports both molecular generation and understanding. The core of Omni-Mol is our proposed MoGE, which dynamically adapts to the intrinsic rank of different tasks. This mixture-of-experts architecture enhances the model's ability to handle diverse tasks and modalities effectively. Our model achieves unified instruction tuning across 16 tasks and attains state-of-the-art performance on 13 of them. Extensive experiments further demonstrate the scalability and versatility of Omni-Mol.

Omni-Mol: Multitask Molecular Model for Any-to-any Modalities
Chengxin Hu!
Hao Li Yihe Yuan Zezheng Song
Chenyang Zhao*
Haixin Wang*
National University of Singapore
Independent Researcher
3 University of Maryland, College Park
*University California, Los Angeles (e13242688u. nus edu, whx@ucla.edi edu]
GitHub: Omni-Mol-Code
HuggingFace:
Ommi-Mol-Data&' Weight
Abstract
In the molccular domain, numerous studies have explored the use multimodal large language models (LLMs) construct general- purposs,, multi-task molec- ular model. However, these efforts are still far from achieving a truly universal molecular model. We identify three key challenges in this endeavor: (1) Existing molecular task datasets are typically small in scale and lack comprehensive domain coverage. (2) Tasks from different molecular subfields are difficult to effectively learn jointly through LLMs due to significant distributional shifts and competition among tasks, which introduces instability in the learning process. (3) Both inter- task and intra-task molecular representations demand different intrinsic dimensions in the language space, making it challenging to balance between redundancy and insufficiency in language model representations. To address these challenges, we innovatively categorize existing molel-mo tasks into four types: Mol2Mol, Mol2Text, Mol2Num, and Tex12Mol. We then collect a dataset encompassing over 16 tasks with more than 1.4 million samples, making the largest molecular instruction-tuna tuning dataset to date. Leveraging the extensive pretraining of LLMs on existing chemical literature, we propose novel multimodal LLM framework, named Omni-Mol, which unifies all small-molecule tasks and supports both molec- ular generation and understanding. The core of Omni-Mol our proposed MoGE, which dynamically adapts to the intrinsic rank of different tasks. This mixture- of-experts architecture enhances the model's ability to handle diverse tasks and modalities effectively. Our model achieves unified instruction tuning across 16 tasks and attains state-of-the-arr performance on 13 of them. Extensive experiments further demonstrate the scalability and versatility of Omni-Mol.
1 Introduction
Large language models (LLMss, especially multimodal LLMs, have achieved significant break- throughs in various scientific tasks due to their powerful representational capabilities and general reasoning abilities, spanning domains such as medicine 22) 82). chemistry (6. and biology [77].
This cutting-edge technology has also sparked an increasing number of studies exploring how to
align molecular representation spaces with textual representation spaces [10 13 21 [1] 25). These
works hold great promise to build powerful AI chemists for advancing molecule captioning, proper- ty/structure prediction, and rext-conditioned de novo drug design.
Eqqual contribution, listings are alphabetic. Coorespponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Benchmark
No LLM Pre training 2. One model lits all 3. 3D uduptive ability
4. Data scaling
5. Parameter scaling 6. Task type
InstructMal [T0
HIGHT [13
X
X
3
LT]
3D-Mol M [4]
X
X
ReactXT 48
X
Omni-Mol
V
Table 1: A comprehensive comparison between Omni-Mol and other molecular LLMs.
The first step in creating an AI chemist is to develop a generalist model with universal capabilities, enabling it to understand diverse molecular structures and their interactions under multiple chemical domains. Pioneering works, such as Text+Chem T5 14, introduce the first multi-domain, multi-task language model capable of unifying molecular and textual representations. Following this, the recent state-of-the-an PRESTO [I]] further enhances performance by progressively improving multimodal LLMs through cross- cross-modal alignment and multi- molti-graph understanding.
From this trend, it can be seen that the community is pursuing a "one-modellfit-all" 83 79 paradigm, rather than using models like InstructMol (10) that rely different Low-Rank Adapters(LoRA) 26 to learn different tasks.
However, existing approaches remain far from achieving a truly general-purpose molecular model. For instance, while PRESTO can generate molecules given molecular inputs or predict properties based molecular structures, it does not support tasks such describing molecules designing molecules according to specified textual requirements. To date, it is rare to find a model that supports sufficiently broad range of task types under unified mone-moee--fis- three main challenges in constructing such a universal molecular model. First, existing molecular task datasets generally small in scale and lack coverage across diverse domains. Second, molecular tasks from different domains exhibit significant distributional discrepancies, making it difficult for LLMs to learn effectively and stably across tasks. Finally, both intra-task molecular instances and inter-task representations differ in their intrinsic dimensionality within language space, making challenging for the model balance redundancy and insufficiency. These issues impede the development of a general-purpose AI expert for molecular tasks.
framework. We identify
In this paper, we seek the answer to the following question:
it possible to develop generalist molecular I.M capable of effectiveey learning across diverse task domains?
This question drives us to develop Omni-Mol, a scalable and general-purpose Multimodal LLM- based framework for unified molecular understanding and generation. Omni-Mol provides four key innovations. (1) we conduct a comprehensive investigation of small molecule tasks and innovatively categorize these tasks according their input-output modalities into four types: Mo12Mol, Mo12Text, um,2Num, and Text2Mol. Subsequently. we construct the Omni-Mol dataset, which comprises over 1.4 million samples and represents the most extensive instuuction-tunigg dataset for small molecule tasks to date. (2) Leveraging this dataset, we propose unified instruction tuning paradigm and build the most comprehensive general-purpose purpose multimodal molecular LLM based on LLaMA [ for the first time. (3) To address the challenge of varying intrinsic dimensions across different domains and tasks, we propose Gradient Adaptive LoRA (GAL), novel adaptive mechanism that extends 67 to better handle multi-task learning scenarios. GAL mitigates conflicts that arise when standard LoRA struggles to accommodate dynamically shifting intrinsic dimensions during training. (4) To further manage inter-task and cross-modal interference, we adopt a Mixture-of-Fe of-eeerts (MoE) framework to develop a -faattrr--o--L--Epppeee (MoGE) fine-tuning strategy. By integrating shared experts and routed experts, our model is capable of both robustly capturing general knowledge and differentiating across diverse tasks.
existing LoRAs 226 17 68, 74| 73
Comprehensive experiments on our datasets show that Omni-Mol achieves significant improvements across 13 tasks simultaneously, setting new ofate-of-the-art results among both finetuned open- source LLMs and in-contex, learned closed-source LLMs. Additionally, we observe that Omni-Mol scales effectively with increases in data volume and model size, indicating the model's tremendous potential under larger computational budgets. Furthermore, by analyzing the representations of models trained on progressively more tasks, we discover that the representations become increasingly similar as the number of tasks grows. This provides robust evidence that the model is learning general

---

Tack
#Sanpies
688,725
Tink
ASamples
20
Retrogy
Poeward
Resgen:
129,684
125,384
125,391
TPSA
Weight
HO.LU.
263,100
412,231
13,979
20
100 20
503
00
25
5-
75
MolEdii
Catalosa
222,287
11,091
74,892
Yield
Loge
9,715
Polizfext
Taok
247.652
#Kamples
Twxt2Mol
Tisk
72,559
#Samples
Molccp
DQA
Experiment
32,979
66,335
MeilDssgg 14,984
57,573
6c) Motecclaar Weight
Idi Dorks Complixity
Domain and tasks
(b) Molecule statistics
Table 2: Task statistics, HO.LU.: HOMO-LUMO prediction, I2S: IUPAC2SELFIES. The total molecule data volume reaches 1.42 million.
Figure I: The sunburst chart of Omni-Mol dataset tasks and the statistics the molecules. Our dataset exhibits substantial diversity across several molecular attributes, including atom count, ring count, molecular weight, and Bertz complexity.
representations effectively. We hope dataset and model can pave the way for the community build more powerful generalist Al chemists.
Related Works
2.1 Molecular Foundation Models
Researchers are trying to leverage the world knowledge embedded LLMs to build higher-quality molecular representations by fine-tuning task-specific instructions. Mol-Instruction [21] pioneers the instruction fine-tuning dataset, demonstrating the potential of LLMs in molecular modeling. Subsequently, InstructMol (10) introduces 2D graph features of molecules based SMILES [69],
showing that LLMs can also enhance performance by aligning and fine-tuning their understanding of graph-basee basph features. Soon after, 3D-MoLM [41] explores the advantages of 3D molecular representations in multimodal LLMs, while HIGHT [3] investigates the impact of multi-level 2D graph features molecular understanding. More recently, PRESTO [II] enhances LLMs' comprehension of molecular- related knowledge through extensive domain -speiffc pretraining across eight tasks.
2.2 Unified Generative Modeling
The GPT models 7 have achieved unification across all text-based tasks through large-scale pretraining and instruction tuning. Subsequently, the community has successfully constructed models that can understand data from multiple modalities and simultancously perform tasks related to different modalities by converting features from each modality into tokens 40 recently, the community has also been exploring unified understanding and generation, allowing This development is driving models towards convergence into truly general purpose model capable of solving all tasks. (27) suggests that as models grow more powerful and general, their representations tend to converge, approaching universal space that reflects the fundamental laws of the world. This insight inspires us to explore whether a multi-task generalist also exists in the molecular domain.
16
44 More
models not only understand multimodal data but also to generate multimodal data 83
79
general-purpose
3 Overview
Omni-Mol a multimodal LLM framework handle K understanding and generative molecular tasks simultaneously. It comprises a language model, graph encoder fg. and a projector Sp. The inputs include a text instruction X,, a SELFIES string Xss and the graph data X corresponding to the input molecules, where X is converted from X using RDKit. We model the response Y the

---

probability of the next token as:
, Ha) II Po Po X Xs, Ho, Ya)
where HG = Sf(sf (X6)), and the parameter of the LLM. The graph encoder encodes the molecule graph into its representation hg E Rnxd, where n is the length of the representation. the projector then projects its dimension the LLM's hidden size and obtain He RRxd,, The overview is shown in Figure 2
The complete multimodal architecture details are in Appendix C
4 Omni-Mol Data Curation
The first step in constructing Omni-Mol is the collection of comprehensive and diverse data. In our review of existing work, we identify wide range of chemical tasks, whose inputs and outputs can generally be categorized into the following modalities: Text, 1D molecular sequences, and Tabular numerical value. For instance, a typical chemical reaction task involves mapping input molecules to another molecule, which falls under the category of molecule-to-molecule tasks. Some studies [0] incorporate graph neural networks (GNNs) to encode molecular graph information input; however, graph features are typically not used outputs.
After a systematic review, we categorize the tasks into the following four major types: (1) Mo12Mol, (2) Mo12Num, (3) Mol2Text, and (4) Text2Mo1. We observe that existing works are not yet capable of achieving learning across arbitrary modality pairs. For example, PRESTO covers Mol2Num and Mo12Mo1 tasks but lacks support for Mol2Text and Text2Mol. Similarly, InstructMol and 3D- MoLM include MMm2Num, Mo12Mol, and Mol2Text tasks, but do not support Text2Mol. Omni-Mol will be trained across all four types of tasks.
Data format. To this end, we construct unified instruction-tuning dataset, standardizing the data format as follows: (1) A unique, clear, and concise instruction; (2) A 1D representation of the molecule (no molecule input for Text2Mol tasks); (3) The corresponding task output.
SELFIES v.s. SMILES. Both SELFIES 337 and SMILES 69 are ID modalities for representing molecules text. In the Omni-Mol Dataset, opted for the SELFIES representation. We observed that SMILES strings may fail to be recognized by RDKit [38;; this issue is particularly pronounced in SMILES generated by LLMs. In contrast, syntactically correct SELFIES string can be robustly decoded into a valid molecule 377. This reliable decoding is necessary to guarantee the conversion to a 2D graph, which in turn ensures the generation of 3D graph. These 2D and 3D graph representations are critical for downstream tasks such as molecular docking and conformer generation.
Preprocessing. After collecting the data, we perform series of preprocessing steps and establish comprehensive metrics for molccular LLMs on the selected task. We also look into tasks that are similar to each other and remove samples that are potentially data leakage. To further investigate the understanding of 3D molecules by LLMs, following 3D-MOIT [11. we preprocess the Omni-Mol data using RDKit to obtain the 3D representation graphs of the molecules. Detailed information can be found in Appendix B
Unified Encoding. We encode data from different modalities uniformly into tokens. For molecular ID representations, textual data, and tabular numerical data, we convert them into character sequences and tokenize them using the tokenizer and word embeddings of the LLM. For molecular graph representations, we utilize the node embeddings generated by a GNN as input tokens. Since the lengths of molecular sequences and graph embeddings vary, apply padding to standardize their shapes into a uniform tensor format, thereby enabling parallel training.
Dataset Statistic. We conducted a comprehensive statistical analysis of the entire dataset, including metrics such as atom count and ring count. The task types along with the names of individual tasks in Figure [ The final Omni-Mol dataset contains over 1.4 million samples, providing ample training fuel for the development of Omni-Mol. The detailed information on each task and the chat template can be found in Appendix B Moreover, we introduce MolEdit as a novel task within our collected data. As the original data was not formatted for instruction- followiig,......ingg we designed and authored specific instructions for molecular editing.

---

MolaMal Task
Given Becrattet and roaazza dhe isoduct
R-2
[C]ce[ [[C [unss]]
Faceder
[CC]JI Ofc[Boncl)...
Rester
Graph Tokminr
MollNom Nam Task
Deiomize de mntecalaa wiiats
(Dnrcch [[CC)
SELFIES
The meleorar meleararsegnt seights
ginl
Instraction
GAL Layer
Mol2Test Task
Can you describe unlecule?
Llama3
MIIA
[C]]C[]Beacck][CC [[CCCOO BOU.BOBBBB1111100 (82--1
Stage
Stage ExtesteT
The unterale is onganopansphate ouverice...
Algeneoo
Adeptive Gradieat aRA
Prperiladd
1
TextZhal Task
Freirered
you dedga this nolecull!
Projeter
The nelocde is regarpphoppatt conariio....
Projeter
Graph
Gragh Excader
Figure 2: Overview of our proposed Omni-Mol, a scalable and general-purpose molecular LLM-based framework for both understanding and generation.
5 Method
5.1 Gradient Adaptive LoRA
In standard LoRA framework, the update AW of the model's weight Wo is defined as: W Wo + AW Wo + ABA, where A and are low rank matrices. As defined in the original paper of LoRA [26], the scaling factor depends on the rank of the LoRA, i.e. >r)) a/r, where a syper-parameter that controls the overall update magnitude of the low-rank adaptation and balance adaptation capacity and training stability.
We conduct experiments on tasks across different domains in an attempt identify an appropriate optimal rank. As shown in Figure 3] we observe that the performance of each domain-specifie task varies under different rank settings, and the optimal rank also differs across domains. For instance, the optimal rank for forward prediction is 128, whereas for molcap it is 32. We attribute this to the differences in the intrinsic dimensionality of attention weights across tasks (2]. Therefore, employing static adapter to simultaneously learn multiple molecular tasks may be suboptimal.
During multi-task training, certain tasks provide highly informative signals, whereas others contribute predominantly redundant information. Consequently, is desirable to introduce a dynamic coefficient that adaptively amplifies the gradients associated with each task throughout the training process.
We propose an adaptive adapter, Gradient Adaptive LoRA (GAL), which introduces a dynamic scaling factor to modulate the fusion of the updated weights, To a/r" + B, where are learnable parameters. Here, the p exponent lets us model rank effects and 3 can provide direct adjustment the scaling factor. This simple yet effective modification enables the adapter to dynamically adjust its scaling factor during training, allowing it to better adapt to the intrinsic dimension of the data.
{a, p, B}
Through this approach, we can dynamically adjust the amplitude of the gradient. The backward propagation of LoRA the downstream fine-tuning data D is modified as:
Vaw Oc (D; Wo + AW) VAW ac (D:Wo W. 78
AW)
dAW
dAW
Implementation details are provided in Appendix
5.2 Mixture-of-GAl--Expaa (MoGE) Expansion
Omni-Mol needs to learn a wide range of different tasks and handle multiple modalities, including graph features, text, and SELFIES. While SELFIES is treated as regular text input to the LLM, it inherently differs significantly from natural language semanties, requiring the model to separately leam how to understand and generate SELFIES expressions.
5

---

Forward
Molcap
LogP
MolDesigu
0.360
0.210
0.570
0.358
Mean
0.205
1.00
0.566
0.354
0.200
16 52
64
128
16
32
64
128
16
32
64
128
16
32
128
LoRÁ rank
LoRA rank
LoRA rank
LoRA rank
Figure 3: Evidence of varying intrinsic dimensions across spec-specific representations. We observe that the optimal LoRA rank (indicated by the yellow triangle) differs across tasks.
We aim for the model to simultancously learn general knowledge while also differentiating different modalities and tasks. Hence, we propose Meatttrr-of-GL--Epeeeee (MoGE). We borrow the idea of MoE [T5] and perform upcycling [36 42] with the aforementioned GAL. We first construct V routed experts, each targeting specialized knowledge areas, and dynamically balance conflicting signals among these experts to effectively mitigate task-level conflicts. Besides, we introduce an additional shared expert learn the common knowledge that underpins fundamental understanding across tasks, by consistently capturing and aligning shared features to maintain stable global representation.
Specifically, we modify the Multi-Head Attention(MHA) and Feed Forward Network(FFN) layer as:
h1 h4-1 MHA (LN(h)-1))
1/= h1-1 + MHA (LN(-(1)))))) h = h1 + (LN(())()))
1 + FFN (LN(M)) 1 1...... IMOGE
h1
14 + MoGE (LN(K)),
1= (MoGE
where h is the hidden states of the l'th layer, Φ, P are the pre- parameters of the LLM, LN represents the norm layer of the LLM. We wrap the parameter p' p GALp. o' 6 GAL For MoGE layer, we initialize N experts with the weight of the pre-trained FFN Here, concludes A routed experts learn specialized knowledge and 1 shared experts learn the common knowledge. Let be the parameter of the i-th expert, and at the beginning of the training, these experts have identical weights, i.e., P1 P2 Kaiming uniform [24], where vp is the learnable parameter of the router.
p. Router Ry is randomly initialized with
5.3
Training
The training strategy of Omni-Mol consists of two stages.
Stage 1: We perform multimodal alignment PubChem [33] learning to describe molecules through graph modality features. The input consists of instructions and graph data, excluding SELFIES. Only the multimodal projector fp is trainable.
Stage 2: We fine-tune Omni-Mol by freezing the pre- trained parameters (wrapped by GAL), while the adapters, expert router, and the multimodal projector stay active throughout fine-tuning
Training loss of both stages language modeling is:
LIM - - X, Ho, Y
For stage 2, we incorporate an additional auxiliary load balancing loss for the MoGE layers, assume an input tensor z E Bxxxx xd and & experts out of N is selected, the load balancing loss is: Laux
# D. Mn Cij 5.g. where C₁ #Y DTE 1{t'th token selects expert i}, 5.j # ΣT Si and 1()) is indicator function. Here, 8i.i. et is the router logit of the t'th token for j'th expert in batch i. This load balancing loss used in [3] additionally considers the sequence-level information.
VV
The total loss is combination of LIM and Laxx with coefficient A: L = LLM + ALaxx

---

Model
#Pu
Exa
BLEU
Ley
RDX
MMC
Mor
Val
Model
FFar
1550
BLEU
Ley
RDK
MAC
Mor
Vil
Forwant Reaction Prodiction Trikk
Retnsyythess Task
DecgScekV3
SB
ICL.
0858
0.35
0.929
12.76
0,719
0,825
0.68
1.00
DeegSeeeiV3
[0
JCL.
0858
0.29
0.93
14.32
0,725
02827
0.68
1000
65
S1.
6.78
0.01
0.804
29.95
0.499
0.649
0.41
1.00
Lluma22
13
6.78
8.00
0.283
53.51
0.136
0.794
0.11
1.00
Mol-Ins
81
St.
6.78
01015
0.654
27.26
0,315
0.509
0.26
1.00
Mal-Ins
21
SL
6.78
0.01
0.705
31.25
0.283
0.487
0.23
1000
HIGHT
SL
6.78
0.29
0.955
16.59
0.774
0.615
0.57
1.00
HIGHT
6.78
0.914
20.20
0.772
0.623
0.58
0.9
InstructMol
10
St.
6,78
0.54
0.967
10.85
0.776
0,878
0.74
1.00
Instructoo
10
SL
6.78
0.41
0.941
13.97
0.753
0,852
0.71
100
PRESTO"
GL
3.28
0.766
6.53
0.871
0.84
PRESIO"
3.28
0.558
0.823
0.887
1.00
Omai-MMl
GL
2.20
0.73
0.960
5.55
0.895
0,947
0.87
1.00
Omni-MMl
GL
2.28
0.57
0.960
8.97
0.864
0,909
0.83
1.00
Model
FPu
Exa
BLEU
Ley
RDK
MAC
Mox
Val
Model
6Par
Exa
BLEU
Lev
RDK
MAC
Mor
Vil
Reagent Prodiction Tock
Gaalyst Prodiction
DeepSeekV3 80
ICL
6858
0.26
0.684
15:20
0.501
0.581
0.47
0.99
DeepSeekV3 63
ICL
685B
0.28
0.189
7.83
0,510
0.481
0.29
0.99
Lamaa
[3
S1.
6.78
0.383
53.51
0.1%6
Vicuna v1.5
120
0.853
0.869
Mol-Ins 81
S1.
6.78
0.04
0.224
23.17
0.237
0.364
0.21
1.00
mechi-bise
8.00
0.07
36.44
0.129
0.055
0.01
0.85
HIGHT
1
6.78
0.487
27.17
0,462
0.30
Mol-tas 20
0.031
0.045
0.02
InstrecMol
10
St.
6.78
0.13
0.610
19.06
0.444
0.539
0.40
1.00
TSChem
Gl.
0.022
0.346
13.41
0.146
0.268
0.20
0.99
PRESTO"
GL
3.28
0.21
0.712
16.31
0.544
0.607
0.48
1.00
PRESTO
GL 6.78
0.77
0.814
0.914
0.895
0.77
100
Omni- Mul
GL
2.20
0.23
0.726
14.59
0.557
0.627
0.52
1.00
Omni-MMl
GL
2.28
0.72
0.792
1.96
0.901
0.855
0.72
1.00
Modal
Type
SParam
HOMO
LJIMO
GAP
Avg.
Model
Tye PPuunm
B-2
B-4
R-1
R8-2
R8-1
Quantum Mechanics Property Prodiction Tisk
Molecalar Captioning Tisk
Vee3
JCL
685B
0,0200
0.0599
0.0457
0.0456
DeepSeekV3
ICL
6850
0.181
0.095
0.319
0.133
0.249
0.251
LLaMA2
ICL.
6.7B
0.7367
0.8641
0,5152
0.7510
GPT-4-0314
RT
0.607
0.525
0.634
0.476
0.562
0.610
Vicuna
ICT.
138
0.7135
3.6807
1.5407
19733
RioMatGPP
108
0.234
0.141
0.386
0.206
0,332
0.208
Mol-Ins
SL.
6.7B
0.0210
0,0210
0,203
0.0210
Mol-Ins
St.
6.78
0.349
0.171
0.331
0.203
0.289
0.271
HIGHT
S1.
6.78
0.0056
0.0065
0.0077
HIGHT InstructMol
6.78
0.4%8
0.397
0.582
0.414
0.518
0.525
Instructhiol
SL
6.71
0,0044
0.0050
0.0061
0,0050
SL
6.78
0.475
0.371
0,506
0.394
0.502
0,,00
Ohani-Mol
G1.
2.28
0
0.0047
0.0049
Omni-MMl
2,28
0.529
0.440
0.604
0.447
0.541
0.571
Model
APar R-2
Medel
APar
B-H S-M
Model
uPar
Weight
LogP
TPSA
Descriptinn (JAA Task
Vield Prediction
More No12
DeepSeckV3
0.39
0.54
DeepSeekV3
685B
-0.55
-1.09
DeepSeekV3
6850
98.63100)
2,26(100) 60.32(100)
Lloma2
SL 6,78 0.28 0.23 0.35
0.22
0.30 0.47
Lluma2 SL 6.7B
-0.48
0.12
Lhmu2
St. 6.78
(96) (.45(95) 15.87 (92)
3D MoLMiS)
0.52
Vicuna
0.15
McLM(S)
6.78
(95) 0.66 (97) 9.71 (93)
3D MDG
GL 6.78 0.32 0.26
0.40
0.26
0.35
0.52
PRESTO GL
6.78
0.94
0.65
3D- MD-MoLM(()
GL 6.78
16.55 (92) 0,78 (95) 10.90 (90)
Omni-Mal
2:28
0.58
Omni--Ma
2.28
0.94
0.68
Omni-Mol
11,07(100
0.49(100)
Model
#Par
Exa
BLEU
Lev
RDK
MAC
Mor
Model
B-2
B-4
R-2
R-L
Solvent Prediction
Expesimental Procedure
DeepSeckV3
ICL
685B
0,06
0.471
5.08
0.196
0,248
0.15
0.99
DeepSeckV3
ICL
685B
Vicuna v1.5
SL.
6.78
0.32
0.436
3.81
0.459
0.486
0.43
1.00
ToaChemiT
GL
220M
0.541
0.406
0.615
0,403
0.564
nachöl base
0.00
0.022
36.44
0.129
0.055
0.01
0.85
MoUTS Large
780M
0.545
0.410
0.625
0.409
0,572
Mel- Instuctioon
SL
6.7B
0.00
0.155
25.12
0.030
0.122
0.01
1.00
Galatica
SL
1.38
0.535
0.395
0.608
0,385
0.552
T5Chem
0.08
16.22
0.458
0.424
0.40
0.99
MolCA, Galac
1.38
0.549
0.415
0.404
0,570
PRESTO
GL
6.78
0.42
0.695
2.76
0.529
0.547
0.51
0.91
ReacXT,, Galae
SL
1.38
0.574
0.460
0.644
0.427
0.589
Mol
GL.
2:28
0.52
0.759
2.71
0.671
0.673
0.64
1.00
Mol
G1.
2.28
0.572
0.448
0.533
0.274
0.464
A 2 0.4
Model
QED
drd2
QED 2 0.6
A 0.4
A 2 0.4
drd2 2 0.5
QED 2 0.6
dri2 2 0.5
dd12 2 0.5
Molecuie Esliiig
Llama3. 2 IB
SL
1.2B
0.8846
0,0507
0
0.4481
0.0260
0.0271
V3
ICL
685B
0.8750
0.0023
0,7500
0.0017
Omni- Mol
GL
2.28
0,9612
0.0653
0.0412
0.7913
0.0560
0,431
Table 3: Comprehensive comparisons on Mo12Mo1, Mo12Text and Mo Kum tasks. Par: Parameters,
Lev: Levenshtein, MAC: MACCS, Mor: Morgan, Val: Validity, Avg.: Average B-2: BLEU-2, B-4:
BLEU-4, R-1: ROUGE-I, R-2: ROUGE-2, R-L: ROUGE-L, M: METEOR, ICL: In-Context Learning, SL: Specialist, GL: Generalist, RT: Retrieval. * means our re-implementation. 3D-MoLM((S) is the specialist version and 3D-MoLM(G) is the generalist version.
6 Experiments
We aim to address the following concerns: (1) Compared with existing baselines, can Omni-Mol achieve the best performances on the comprehensive omni-molecular datasets with 16 tasks? (2) Is Omni-Mol a scalable framework with the capacity and potential to solve complex molecular tasks? (3) Are all key components of Omni-Mol essential for omni-molecular task learning? We begin by describing the experimental setup and then answer all the questions in the subsequent sections.
6.1 Experimental Setup
Baselines. To ensure fair comparison, we first choose representative L.L.-based models such as InstructMol and HIGHT, and also report several previous baselines, including Mol-Insstuctton.
7

---

1.1
Avsagge Maleap
Borwal
Beagatt Hemoium
Performance
0..8
Malop
0.6
Performane
0.75
0.5
Training (%)
Figure 4: Scaling trend of Omni-Mol. (Left) The scaling trend with respect to dataset proportion,
metrics are averaged across tasks. (Right) The scaling trend with respect to model size.
BLEU
RDK
MAC Mar
JUPACISELFIES
Test Guided Molecule Generation
L
0.477 0818
Llama3
0.04
DeepSeek
ICL 6850 0.00 0.828
20.37 0.177 0.399 0.14 0.893
DeepSeek
ICL 685B 0.02
0.658
35.27
0.217
0.398 0,170 0.608
Omni-Mul
Ommi-Mell
0,221
Table 4: Results of Text2Mol tasks, # means we train the model with LoRA and the same multimodal configuration as Omni-Mol.
Llama, and Vicuna 80). We also conduct shot in-context learning test on powerful open-source models like DeepSeek V3. For datasets with fewer models, re-implement PRESTO as baseline.
Backbone. utilize LLaMA 3.2-IB [18 as the backbone, a single linear layer as the projector, and MoleculeSTM [54] the graph encoder. For MoGE expansion, we set IMGGE = 1/4L and the number of experts 5, there are 2 routed experts and shared expert in total. More details about model implementation can be found in Appendix C
Evaluation Metric. Following [T] we evaluate Mo12Num tasks with MAE and R². For Mol2Text we adopt the standard NLP suite, which are BLEU-2, BLEU-4, ROUGE-2, ROUGE-L, and METEOR
outlined in T0 41). Text2Mol and reaction-related Mol MMolMol tasks, following 10 78 are gauged with Exact Match, Levenshtein score, MACCS similarity, Morgan similarity, and RDK similarity, quantifying how ID molecular strings encode functional and structural information. For the molecule editing task in Mo12Mo1. we report both unconstrained and constrained success rates. The constrained variant requires (i) the optimized molecule to satisfy the property thresholds, which are QED (Quantitative Estimate of Drug-likennes) S > 0.6 and DRD2 score (probability of being active DRD2 ligand) (55] 0.5, and (ii) a 2D topological similarity to its precursor at least A = 0.4. QED is computed with RDKit. All of evaluation metrics take into account both the linguistic quality and their biological relevance. Details are in Appendix
Training Details. We use PyTorch (66] with DeepSpeed ZeRO-2 [58] For unified tuning, we train 15 epochs with GAL rank of 64. For separate tuning, model is trained for 10 epochs with the same GAL configuration. The learning rate is set to 8e-5 from the grid search for all experiments. For consistency, the random seed is set to More details can found Appendix
6.2 Main Results
Here, we obtain the answer that Omni-Mol can achieve the best performance across almost all tasks. As the results shown in Table we have the following observations. Omni-Mol significantly outperforms almost all specialist baselines while utilizing only 33% of the parameters. Furthermore, Omni-Mol surpasses the corresponding state-of-the-at generalist baseline by an average of approxi- mately 5%, 7%, 9%, 11 %. and 40% across forward prediction, retrosynthesis, reagent prediction, molcap, and Description Q&A separately. We further notice that on Mol 2Num tasks, Omni-Mol im- proves the R² score by 21.4%, and lowers the Mean Absolute Error(MAE) by 25.1%. 25.8%, 39.3% on Weight, LogP and TPSA regression respectively. That is to say. Omni-Mol achieves superior performance with greater parameter efficiency, demonstrating its effectiveness becoming general Al chemist. To further examine GAL's adaptability to 3D molecular tasks, we report Omni-Mol' results on ten tasks. Omni-Mol achieves the best performance on all ten tasks, outperforming the above basclines. Additional experimental details are provided in Appendix E

---

Avaragua Parformancc
1.0
1.0
0.56
3.82
6.8
et karrey
0.50
0.52-
3.80
6.6
Avg Pert
Avg Pert
0.50
6.6
0.48
9.78
0.2
0.46
9.76
0.44
melccp
Hllld
MolDcsign
reagent
molcap
Vieid
MolDesign
reagant
mdcapp
yleid NoiDwsigs
0.74
Figure 5: Ablation studies: (Left) Averaged performance comparison, the results demonstrate that InstructMol with joint training outperforms separate training. Further integrating MoGE yields additional performance gains across all tasks. (Middle) Ablation of Gradient Adaptive LoRA. (Right) Ablation of MoGE expansion. For the mid and right figure, the left axis represents the average performance of Reagent, Molcap and MolDesign. The right axis represents the average performance of Yield Prediction.
6.3 Is Omni-Mol a Scalable Framework?
One critical property of LLMs is their scaling behavior in relation to both model and data size. In this study, we demonstrate that Omni-Mol a scalable framework by conducting two distinct types of scaling experiments: (1) We select three different sizes of LLaMA 3 series, IB, 3B, and 8B, for language backbone scaling. More backbone results are in Appendix (2) We evaluate the impact of dataset size by down-sampling the original dataset to 20%, 40%, 60%, and 100% of its full size.
(1) Data Scaling. As shown the left of Figure we observe clear scaling trend as the dataset proportion increases, indicating that the model's performance improves as the amount of data increases. This suggests that further increasing the dataset size can bring more benefits and build stronger and more generalized chemical AI.
(2) Parameter Scaling, As shown the right side of Figure the performance of Omni-Mol across all tasks increases as the model size grows. We also observe clear scaling trend. However, this trend not pronounced as the performance gains resulting from increased dataset size, which suggests that there remains potential for further expansion in the amount of data.
6.4 Is Unified Instruction Tuning Essential?
We evaluate whether our Omni-Mol dataset can enhance unified instruction tuning. We select one task from each of the following categories: Mo12Mol, Mol2Text, Mo12Num, and Text2Mol, and retrain InstructMol on each. We compare the results of individually trained models (one LoRA per task) with those trained a unified manner (a single LoRA shared across all tasks), as shown in Figure The results indicate that unified learning on the Omni-Mol dataset yields performance improvements. Interestingly, the moleap and yield tasks exhibit noticeable differences from the other tasks, yet still benefit from unified fine-tuning, with yield regression showing the greatest performance gain.
6.5 Ablations MoGE
How do MoGE help? Building upon the unified training of InstructMol on our dataset, we incorporate MoGE into the framework. As illustrated in Figure InstructMol with Joint Training and MoGE consistently outperforms both InstructMol with Separate Training and InstructMol with Joint Training, demonstrating the effectiveness of MoGE. By adapting the intrinsic dimensionality of different tasks and leveraging the specialization among MoGE experts, MoGE enhances Omni-Mol's ability to generalize across a diverse range of tasks and modalities.
Is GAL essential? We compare our Omni-Mol with Omni-Mol w/o GAL, which replaces the Gradient Adaptive LoRA with the standard LoRA. As shown in the middle of Figure 5 Omni- Mol w/o GAL consistently exhibits lower performances than Omni-Mol. This consistent decline underscores the effectiveness of our GAL in enhancing performance by adaptively adjusting itself to the intrinsic dimension.
Is MoGE expansion essential? We conduct ablation study by removing the MoGE expansion and training the model with GAL alone (Exclude MoE). The comparison results are shown on the right side of Figure We observe that Omni-Mol consistently outperforms the Omni-Mol w/o

---

MoE across multiple tasks, including reagent prediction, molcap, yield regression, and text- exxt-guided molecule generation. These results demonstrate that Omni-Mol effectively enhances performance by leveraging specialized experts. The most significant improvement is observed in yield regression, where the diversity of experts contributes better generalization and representation learning.
6.6 Convergence Analysis via Mutual Similarity
Omni-Mol is trained across wide range of tasks, we aim to examine how its learned representations vary with respect the number of tasks involved in training.
the model is indeed learning within a progressively smaller solution space, we should observe convergence in representation similarity. This is because, under larger number of tasks, the reduction in the solution space is expected to constrain the variation in the learned representa- tions. We compute the representation sequence with the model trained on 2, 4, and tasks with mutual aatual_knn [27], the results are shown in Fig- ure Obviously, when the number of tasks increases, the similarity of the representations learned by Omni-Mol also increases. This indi- cates that the model's 's representations are grad- ually converging. This outcome supports the hypothesis that adding more tasks reduces the size of the model's s general solution space. Ulti- mately, these representations converge to a uni- versal form that can effectively solve all tasks. Interestingly, in the mutual similarity analysis of InstructMol, we observe the opposite trend. As the number of tasks increases, the represen- tations learned by InstructMol become progres-
S taske boos toek
0.89
0.87
*
0.86
0.85
Omani-Mol
InstreutMal
0.84
Figure 6: Demonstration of similarity scores heatmap for methods trained on varying numbers of tasks.
sively less similar those learned previously. This suggests that with each added task, the changes in the solutions learned by InstructMol become larger, indicating that it is unable to converge to universal representation space through unified training.
7
Conclusion
We introduce Omni-Mol, model that unifies 16 tasks and Omni-Mol dataset, with over 1.4 million samples. Omni-Mol learns generalizable representations and achieves this through unified tuning, MoGE expansion, and Gradient Adaptive LoRA. Omni-Mol achieves SOTA performance across multiple tasks, and we demonstrate its scalability and ability to scale up performance as the number of tasks increases. Finally, we provide experimental evidence showing that Omni-Mol achieves more general convergent solution space, acquiring the general capability to solve diverse tasks.
Limitations
We identified two limitations: (1) Due to the limited computational resources, unable to further scale up the model with higher computational resources, which prevents us from exploring the limits of model's performance. (2) Omni-Mol's tasks are still primarily focused small molecules, future work should explore proteins and the interaction between proteins and small molecules.
Broader Impacts
This paper presents Omni-Mol, which the first scalable and unified molecular generalist model with outperforming results, enabling tasks such as molecule captioning, property prediction, and drug design. While these advancements provide powerful tools for molecular research, they also raise ethical concerns, such as the risk of misuse in designing harmful molecules. Transparency, responsible use, and interdisciplinary collaboration are essential to ensure these models serve the broader good, paving the way for impactful and responsible scientific innovation.
10

---

Acknowledgment
We thank the University of Maryland, College Park for providing us abundant computational resources. We sincerely thank Fanding Xu for his help in verifying the case study. His professional biochemical knowledge was invaluable this process.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 2rxxiv2303.08774. 2023.
[2] Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. Intrinsic dimensionality explains the effectiveness of language model fine-tuning. arXiv preprint arXiv:20/2.1322. 2020.
[3] Derek T Ahneman, Jesús G Estrada, Shishi Lin, Spencer D Dreher, and Abigail G Doyle. Predict- ing reaction performance in c-n cross-coupling using machine learning. Science, 160 190, 2018.
[4] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-23736. 2022.
[5] Richard Bickerton, Gaia Paolini, Jérémy Besnard, Sorel Muresan, and Andrew Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90 98, 2012.
[6] Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes Autonomous chemical research with large language models. Nature, 624((992):570--78) 2023.
[7] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1901, 2020.
[8] Darko Butina. Unsupervised data base clustering based on daylight's fingerprint and tanimoto similarity: A fast and automated way to cluster small and large data sets. Journal of Chemical Information and Computer Sciences, 39(4):747-750, 1999.
[9] Kerim Büyükakyüz. Olora: Orthonormal low-rank adaptation of large language models. arXiv preprint PrXiv:2406.01777 2024.
[10] He Cao, Zijing Liu, Xingyu Lu, Yuan Yao, and Yu Li. Instructmol: Multi-modal integration for building a versatile and reliable molecular assistant drug discovery. In Proceedings of the 31st International Conference Computational Linguistics (COLING 2025), Abu Dhabi, UAE, Jan 2025.
[11] He Cao, Yanjun Shao, Zhiyuan Liu, Zijing Liu, Xiangru Tang, Yuan Yao, and Yu Li. PRESTO: Progressive pretraining enhances synthetic chemistry outcomes. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10197-10224, Miami, Florida, USA, November 2024. Association for Computational Linguistics.
[12] Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd aem sigkdd international conference on knowledge discovery and data mining, pages 785-794, 2016.
[13] Yongqiang Chen, Quanming Yao, Juzheng Zhang, James Cheng, and Yatao Bian. Hight: Hierarchical graph tokenization for graph-language alignment. arXiv preprint arXiv:2406. 14021, 2024.
[14] Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, and Mat- teo Manica. Unifying molecular and textual representations via multi-task language modelling. In International Conference on Machine Learning, pages 6140-6157. PMLR, 2023.
11

---

[15] Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y Wu, et al. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models. arXiv preprint arXiv: 24Xi:2401.06060 160666, 2024.
[16] Wenliang Dai, Junnan Li, D Li, AMH Tiong, Zhao, W Wang, B Li, P Fung, and S Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. arxiv 2023. arXiv preprint Frxxiv23305006000 2, 2023.
[17] Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, pages 1-16, 2023.
[18] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024.
[19] Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. arXiv preprini arXiv:2204. 012204.11811, 2022.
[20] Carl Edwards. ChengXiang Zhai, and Heng Ji. Text2mol: Cross-modal molecule retrieval with natural language queries. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 595-607, 2021.
[21] Yin Fang. Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. The Twelfth International Conference on Learning Representations, 2024.
[22] Janna Hastings, Gareth Owen, Adriano Dekker, Marcus Ennis, Namrata Kale, Venkatesh Muthukrishnan, Steve Turner, Neil Swainston, Pedro Mendes, and Christoph Steinbeck. Chebi in 2016: Improved services and expanding collection of metabolites. Nucleic acids research, 44(D1):D1214DD121. 2016.
[23] Soufiane Hayou, Nikhil Ghosh, and Bin Yu. Lora+: Efficient low rank adaptation of large models. arXiv preprint arXiv:2402. 200.:2242./3344 2024.
[24] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages
2015.
[25] Chengxin Hu, Hao Li, Yihe Yuan, Jing Li, and Ivor Tsang. Exploring hierarchical molecular graph representation in multimodal llms. arXiv preprint arXiv:2411.04700 2024.
[26] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106. 09685, 2021.
[27] Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The platonic representation hypothesis. In Forty-first International Conference on Machine Learning, 2024.
[28] John J Irwin and Brian K Shoichet. Zinc- free database of commercially available compounds for virtual screening. Journal of chemical information and modeling, 45(1):177-182, 2005.
[29] Justin Jee, Christopher Fong, Karl Pichotta. Thinh Ngoc Tran, Anisha Luthra, Michele Waters, Chenlian Fu, Mirella Altoe. Si- Liu. Steven B Maron, al. Automated real-world data integration improves cancer outcome prediction. Nature, pages 1-9, 2024.
[30] Damjan Kalajdzievski. A rank stabilization scaling factor for fine-tuning with lora. arXiv preprint arXiv:23 2023.
[31] Steven M Kearnes, Michael R Maser, Michael Wleklinski, Anton Kast, Abigail G Doyle, Spencer D Dreher, Joel M Hawkins, Klavs F Jensen, and Connor Coley. The open reaction database. Journal of the American Chemical Society, 143(45):18820- 180888808888662 2021.
12

---

[32] Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen. Bo Yu. et al. Pubchem 2021: new data content and improved web interfaces. Nucleic acids research, 49(D1):D1388-D1 395, 2021.
[33]
Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A. Shoemaker, Paul A. Thiessen, Bo Yu, Leonid Y. Zaslavsky, Jian Zhang, and Evan E. Bolton. Pubchem 2023 update. Nucleic acids research, 2022.
[34] Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Generating images with multimodal language models. Advances in Neural Information Processing Systems, 36, 2024.
[35] Walter Kohn and Lu Jeu Sham. Self-consistent equations including exchange and correlation effects. Physical review, 140(4A):A1133, 1965.
[36] Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, and Neil Houlsby. Sparse upcycling: Training mixture-of-experts from dense checkpoints. In The Eleventh International Conference on Learning Representations, 2023.
[37] Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alan Aspuru-Guzik Self-referencing embedded strings (selfies): 100% robust molecular string representation. Machine Learning: Science and Technology, 144):04022, 2020.
[38] Greg Landrum et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum, 8(31. 10):588, 2013.
[39] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730-19742. PMLR, 2023.
[40] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision- language understanding and generation. In International conference on machine learning, pages 12888-129900 PMLR, 2022.
[41] Sihang Li, Zhiyuan Liu, Yanchen Luo, Xiang Wang. Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, and Qi Tian. Towards 3d molecule-text interpretation in language models. In The Twelfth International Conference on Learning Representations, 2024.
[42] Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Junwu Zhang, Munan Ning, and Li Yuan. Moe-llava: Mixture of experts for large vision-language models. arXiv preprint arXiv:2401.15947 2024.
[43] Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint ar 244424405.04444, 2024.
[44] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[45] Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar. Multi-moda molecule structure- text model for text-based retrieval and editing. Nature Machine Intelligence, 5(12):1447-1457, 2023.
[46] Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pre-training molecular graph representation with 3d geometry. In International Conference on Learning Representations, 2022.
[47] Zhiyuan Liu, Sihang Li. Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Molca: Molecular laaph-la modeling with cross-modal projector and uni-modal adapter. arXiv preprint arXiv:23 12798, 2023.
[48] Zhiyuan Liu, Yaorui Shi, An Zhang, Sihang Li, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, and Tat-Seng Chua. Reactxt: Understanding molecular" reaction-ship" via reaction- contextualized molecule-text pretraining. arXiv preprint 142255 2024.
13

---

[49] Micha Livne, Zulfat Miftahutdinov, Elena Tutubalina, Maksim Kuznetsov, Daniil Polykovskiy, Annika Brundyn, Aastha Jhunjhunwala, Anthony Costa, Alex Aliper, Alán Sspuru-Guzie et al. nachO: multimodal natural and chemical languages foundation model. Chemical Science, 15(22):830-88389 2024.
[50] Daniel Lowe. Chemical reactions from patents (1976-se20016)
https
/ //figshare. com wwtiiaaaaaaieeaaaaaaaooaaeeeeeee http://eeeeo.ttttttttttttttt///e hatpaattte/cciillaaaattttttooooo cal 7776-880220166511487
Sarpnns_fro_______ US_ US
2017.
[51] Jieyu Lu and Yingkai Zhang. Unified deep learning model for multitask reaction predictions with explanation. Journal of chemical information and modeling, 62(6):1376-1387, 2022.
[52] Yizhen Luo, Jiahuan Zhang, Siqi Fan, Kai Yang, Yushuai Wu, Mu Qiao, and Zaiqing Nie. Biomedgpt: Open multimodal generative pre-trained transformer for biomedicine. arXiv preprint arXiv:2308. 09442, 2023.
[53] Fanxu Meng, Zhaohui Wang, and Muhan Zhang. Pissa: Principal singular values and singular vectors adaptation of large language models. Advances in Neural Information Processing Systems, 17:12133--111110722 2024.
[54] Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton, and Neil Houlsby. Multi- modal contrastive learning with limoe: the language-image mixture of experts. Advances in Neural Information Processing Systems, 35:9564 9576, 2022.
[55] Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformaties, 9:1-14, 2017.
[56] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, bigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[57] Damith Perera, Joseph W Tucker, Shalini Brahmbhatt, Christopher J Helal, Ashley Chong, William Farrell, Paul Richardson, and Neal Sach. A platform for automated nanomole-scale reaction screening and micromole-scale synthesis in flow. Science, 339(6374)
2018.
[58] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimiza- tions toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-16. IEEE, 2020.
[59] Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Lean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of chemical information and modeling, 52(11) 22(11)226442877
2012.
[60] Philippe Schwaller, Alain C Vaucher, Teodoro Laino, and Jean-Louis Reymond. Prediction of chemical reaction yields using deep learning. Machine learning: science and technology. 2(1):015016, 2021.
[61] Fang Sun, Zijie Huang, Haixin Wang, Yadi Cao, Xiao Luo, Wei Wang, and Yizhou Sun. Graph fourier neural odes: Bridging spatial and temporal multiscales in molecular dynamics. arXiv preprint arXiv:2411.01600 2024.
[62] Ruoxi Sun, Hanjun Dai, and Adams Wei Yu. Does gnn pretraining help molecular representa- tion? Advances in Neural Information Processing Systems, 15:12096-12109, 2022.
[63] Matt Swain. Pubchempy: A python wrapper for the pubchem pug rest api. comm/ncs077PabChhmmry 2024. Accessed: 2024-12-15.
[64] Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arxiv 2022. arXiv preprint arXiv:22/1.09085 10, 2023.
[65] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint 200.::::0070028888 2023.
14

---

[66] USPTO. Fy 2020 performance and accountability report. Technical report, USPTO, 2020.
[67] Haixin Wang, Jianlong Chang, Yihang Zhai, Xiao Luo, Jinan Sun, Zhouchen Lin, and Qi Tian. Lion: Implicit vision prompt tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 5372-5380, 2024.
[68] Haixin Wang, Xinlong Yang. Jianlong Chang, Dian Jin, Jinan Sun, Shikun Zhang, Xiao Luo, and Qi Tian. Parameter-efficient tuning of large-scale multimodal foundation model. Advances in Neural Information Processing Systems, 36:15752-15772 15774, 2023.
[69] David Weininger. Smiles, chemical language and information system. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36, 1988.
[70] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenct: a benchmark for molecular machine learning. Chemical science, 932::513-530, 2018.
[71] Zhenxing Wu, Odin Zhang, Xiaorui Wang, Li Fu, Huifeng Zhao, Jike Wang, Hongyan Du, Dejun Jiang, Yafeng Deng, Dongsheng Cao, et al. Leveraging language model for advanced multiproperty molecular optimization via prompt engineering. Nature Machine Intelligence. pages 1- 2024.
[72] Jun Xia, Lecheng Zhang. Xiao Zhu, Yue Liu, Zhangyang Gao, Bozhen Hu, Cheng Tan, Jiangbin Zheng, Siyuan Li, and Stan Li. Understanding the limitations of deep models for molecular property prediction: Insights and solutions. Advances in Neural Information Processing Systems, 36:64774-64792, 2023.
[73] Bruce XB Yu, Jianlong Chang, Haixin Wang, Lingbo Liu, Shijie Wang. Zhiyu Wang, Junfan Lin. Lingxi Xie, Haojie Li, Zhouchen Lin, et al. Visual tuning. ACM Computing Surveys, 56(12):1-38, 2024.
[74] Yihang Zhai, Haixin Wang, Jianlong Chang, Xinlong Yang, Jinan Sun, Shikun Zhang, and Qi Tian, When parameter-efficient tuning meets general-purpose vision-language models. arXiv preprint arXiv:2312. 12458, 2023.
[75] Qianru Zhang, Xinyi Gao, Haixin Wang, Siu-Ming Yiu, and Hongzhi Yin. Efficient traffic prediction through spatio-temporal distillation. arXiv preprint arXiv:2501.10459, 2025.
(76] Qianru Zhang, Haixin Wang, Siu-Ming Yiu, and Hongzhi Yin. Graph masked autoencoder for spatio-temporal graph learning. arXiv preprint arXiv:2410. 10915, 2024.
[77] Shanghang Zhang, Gaole Dai, Tiejun Huang, and Jianxu Chen. Multimodal large language 2024.
models for bioimage analysis. nature methods, 2
[78] Yikun Zhang, Geyan Ye, Chaohao Yuan, Bo Han, Long-Kai Huang, Jianhua Yao, Wei Liu, and Yu Rong. Atomas: Hierarchical adaptive alignment molecule-text for unified molecule understanding and generation. In The Thirteenth International Conference on Learning Repre- sentations, 2025.
[79] Kaizhi Zheng, Xuehai He, and Xin Eric Wang. Minigpt-5: Interleaved iision-and-languaer generation via generative vokens. arXiv preprint arXiv:2310.02233 2023.
[80] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023.
[81] Gengmo Zhou Zhifeng Gao. Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference on Learning Representations, 2023.
[82] Juexiao Zhou, Xiaonan He, Liyuan Sun, Jiannan Xu, Xiuying Chen, Yuetan Chu, Longxi Zhou, Xingyu Liao, Bin Zhang, Shawn Afvari, et al. Pre-trained multimodal large language model enhances dermatological diagnosis using skingpt-4. Nature Communicationss 15(1):5649, 2024.
15

---

[83] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing ision-language understanding with advanced large language models. arXiv preprint ar0iv:2304. 10592, 2023.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in abstract and introduction clearly match our theoretical and empirical results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
is provided in the Appendix and Conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification:
The complete proof is in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All hyperparameters stated in the Appendix the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer: [Yes]
Justification:
All code and data will be publicly released.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
cessionnPoccyl for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ctpppp..cppulli//guiii wwwiiicgguie///eeeeeeeei CoodeSubeiiieeee ssi wswinnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
They are clearly explained in the Appendix and the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Statistical significance are considered.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
19

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The runtime comparison is in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http://....c..uu.tttttttccuuueee
Answer: [Yes]
Justification: The work conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impacts of this work discussed in the Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
20

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
The datasets and code used in this work do not pose any risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: They are explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee www.reeswthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
21

---

Answer:
[Yes]
Justification:
All datasets and code will be publicly released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
22

---

The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non- standadd components.
Please refer to our LLM policy hattps://neurips... 220p://ee////oofeeeeee/2/22///// for what should or should not be described.
23

---

Appendix
Table of Content
Experiment Environments
Further Details on Datasets
Further Details on Model Implementation
Further Details on Training
Further Details on Experimental Results
More Ablation Study Results
Case Study
Task Definition and Prompt Templates
Discussion on Generalist and Specialist
Discussion on Continual Learning
A
Experiment Environments
In this section, we provide a summary of our experiment environment.
Software and Driver Versions. The experiments are conducted with the following key software
Python 3.12.1
Pytorch 2.5.1
Transformers 4.45.2
CUDA 12.4
Accelerators. Training Omni-Mol costs 576 x NVIDIA A100 80G GPU hours.
B Further details on datasets
B.1
Comprehensive Datasets Construction
In this subsection, we provide a comprehensive list of the datasets used our study along with their respective sources. While datasets vary across different papers, we observed that many are derived and processed from common sources. To clarify this overlap, we summarize the information Table and provide detailed analysis below.
(1) USPTO [66]. The USPTO (United States Patent and Trademark Office) dataset is widely used large-scale chemical reaction dataset extracted and processed from US patent texts. It encompasses diverse range organic reaction types, including esterification, amidation, halogenation, Suzuki cou- pling, Buchwald- Hartwig coupling, addition reactions, condensation reactions, and redox reactions. Following (2I]. for the Forward Reaction Prediction task, we extract data from USPTO, and split the dataset into 124,384 training instances and 1,000 test instances. Partially following [II], for the Catalyst Prediction and Solvent Prediction tasks, we similarly extract data from USPTO, splitting the training/test sets into 10,079/1,015 and 67.009/7,793, respectively.
USPTO 500 MT [51 a high-quality multi-tast reaction prediction dataset, derived from USPTO through manual processing (including data filtering, deduplication, etc.). This subset retains the 500 common reaction types. Following (T], for the Reagent Prediction task, split the dataset into 124,384 training instances and 1,000 test instances.
USPTO 1S0K [51) subset of organic chemical reaction data extracted from USPTO, is widely used chemoinformatics for the single-step retrosynthesis task. Following [2I]. for the Retrosynthesis task, the dataset is divided into 128,684 training instances and 1,000 test instances.
USPTO-Applications (50 is another commonly used subset USPTO, primarily derived from data samples in patent applications. For the Experiment Procedure Prediction task, following [48

---

(along with the introduction of ORD data), we split the dataset into 80% training, 10% validation and 10% test sets.
(2) ChEBI-20 20 ChEBI-20 is derived from the ChEBI-16 22 dataset, with further annotations based on PubChem, forming a comprehensive database of chemical entities in the field of biochem- istry. Compared to [21]. ChEBI-20 provides more extensive and detailed description of chemical compounds. Therefore, for the Molecular Captioning task, following [0], we split the ChEB1-20 dataset (which contains total of 33,010 instances) into 26,420 training instances, 3295 validation instances and 3,295 test instances.
(3) QM9 [70]. QM9 is a subset of the GDB-17 [59 database, focusing on quantum chemical property prediction for small organic molecules. It provides comprehensive quantum chemical attributes for molecular compounds, including spatial geometries and electronic properties, such as HOMO/LUMO energy levels obtained via DFT calculations [5]. In this work, focus on the HOMO/LUMO energy levels of molecules. For the Quantum Mechanics Property Prediction task, following 21, we split the dataset into 360,113 13 training instances and 1,987 test instances.
(4) PubChem [32]. PubChem is the world's s largest open-access chemical information database, focusing on chemistry, bioinformatics, and drug discovery. It provides comprehensive support for the retrieval and analysis of molecular compound data. Partiallyy following [41], for the Molecular Weight Prediction, LogP Prediction, and Topological Polar Surface Area Prediction tasks, we split the dataset into 11,979/2,000, 10,673/1,785, and 11,979/2,000 for training and test sets, respectively. Additionally, for the Description Q&A task, also following 41], we split the dataset into 56,885 training instances and 10,000 test instances. For the IUPAC2SELFII task, we split the training and test sets into 54,811 1 and 2, 7,4 samples, respectively. For the Text Guided Molecule Generation task, we split 11,98622,000 for training and test sets, respectively.
(5) RNX Yields [60]. The RNX Yields dataset consists of the Buchwald- and the Suzuki-Miyaura reaction [57 dataset, both collected through high-throughput experimenta- tion (HTE). It is designed to predict reaction yields for these two reaction types. Following PRESTO, we split the dataset into 9,515 training instances and 200 test instances for Yields Regression.
reaction 3 dataset
(6) ORD [1.. The ORD (Open Reaction Database) is an open-source database dedicated to the standardization, storage, and sharing of organic chemistry reaction data, providing a unified data schema with structured text for organic reaction datasets. Following [48] (along with the USPTO- Applications), for the Experimental Procedure Prediction task, We partition the dataset into 90% for training, 10% for validation, and 10% for testing, based on the total data volume.
(7) ZINC 2 ZINC ("ZINC Not Commercial") is an openly accessible repository of purchasable compounds engineered for structure-based virtual screening. It is committed to offering a no-cost, scalable platform for in silico screens. To ensure direct compatibility with leading docking engines, each entry in ZINC undergoes specialized preprocessing workflow and is furnished with 3D conformers, the number of rotatable bonds, and other routine molecular descriptors. For the Molecule Editing task, we split the training and test sets into 218,708 and 3,579 samples, respectively.
Based on the seven datasets presented above, construct a total of 16 tasks spanning four task types, amounting to 1.4 million data samples. To the best of our knowledge, this represents the most comprehensive dataset date in the small molecular domain.
B.2 Preprocessing
We encounter several issues during processing the datasets, we list them below and elaborate our solutions.
Unable to obtain SELFIES, We retrieve the SMILES representation of molecule with its CID using pubchempy (63] API, for CIDs that cannot be found with pubchempy. Compound. (rom_cid()) we discard them. For molecules that cannot be converted to SELFIES, we discard them.
Overlapped samples. Datasets from different sources often contain overlapping samples, leading to potential data leakage. For example, solvent and catalyst prediction are subsets of reagent prediction, and molecule description data from PubChem [3 may include samples that overlap with those in ChEBI-20 (20). Such overlaps create scenarios where a sample from one dataset's training set appears in the test set of another, compromising the reliability of model evaluation. To address this
25

---

issue, we conduct a thorough dataset comparison to identify potential overlaps and systematicalyy remove any samples from the training sets that also appear in the test sets of other datasets.
From SELFIES to 3D molecule. Compared to the ID and 2D representations of molecules, the 3D) spatial topology reveals richer molecular properties and research value, such as protein interactions and molecular dynamics. Following [41,, we preprocess molecular SELFIES using RDKit. First, we convert SELFIES 2D representation via the selfies library and RDKit and add hydrogen atoms to facilitate subsequent force field optimization. Finally, after embedding each atom at a random initial coordinate, we optimize the resulting conformations using RDKit's 's MMFF94 force field (e.g.. by computing interatomic interaction potentials). Once we obtain the optimized atomic conformations, we execute 3D molecular tasks using the full 3D information.
B.3 Details on Evaluation Metries
Exact Match, The Exact Match Score evaluates whether two SMILES strings unequivocally correspond to the same molecular structure. Specifically, a score of 1 is assigned when both SMILES strings identical following normalization, indicating they represent the same molecule. Meanwhile, a score of 0 is given when the normalized SMILES strings differ, signifying that they correspond to distinct molecules.
Levenshtein Score. The Levenshtein Score scores the smallest number of edit operations needed to transform one SMILES string into another. These edit operations typically encompass: (1) Insertion, which involves adding a character at a specific position; (2) Deletion, the removal of a character from designated location; and (3) Substitution, replacing character at particular position with different one.
MACCS Similarity. Within cheminformaties, MACCS Similarity is used to assess and compare the structural likeness of molecules. This approach is grounded in MACCS keys, which are a standardized set of structural descriptors developed by the Molecular ACCess System. These keys capture and represent essential molecular substructures. To determine the similarity between two molecules, the method evaluates the presence or absence of these predefined structural features.
RDK Similarity. The RDK Similarity generally involves evaluating and quantifying the similarity between molecules by utilizing fingerprints produced with RDKit.
Morgan Similarity. Morgan Similarity is used to evaluate and measure the structural resemblance between molecules by utilizing Morgan fingerprints as its foundational basis.
Mean Absolute Error (MAE). The MAE quantifies the average absolute deviations between predicted results and actual values, which provides straightforward metric for assessing the accuracy of predictive models by averaging the absolute differences across all instances.
R², The R² metric scores the proportion of variability in the target variable that can be explained by the model's predictors. It can serve as an indicator of the model's 's explanatory strength, reflecting how well the observed data points are captured by the regression model.
Unconstrained & Constrained Successful Rate (QED & DRD2). Both unconstrained and con- strained success rates calculate the proportion of predicted molecules that satisfy predefined threshold out of all predicted molecules. The unconstrained variant emphasizes the model's 's ex- ploration of a broader chemical space to produce high-quality molecules that meet the specified properties, such as new scaffolds or substituents with improved characteristics. The constrained vari- considers molecular similarity and focuses on preserving key molecular features (e.g., scaffolds, pharmacophores). QED and similarity are readily computed by RDKit. However, the commonly used approach for computing the DRD2 score relies on machine learning methods (c.g.. support vector machines) for molecular classification (5). Although research shows that non-deep learning methods generally outperform deep learning approaches on molecular classification tasks (72). this finding may not fully account for the impact of deeper-level sample distribution factors, such as class imbalance and disparities in feature distributions on dataset size. Moreover, the generalization capability of non- non-deep learning methods can be limited, which undermines their suitability for DRD2 score computation, since molecules predicted by large language models may be entirely unseen by the DRD2 scoring model. Following [72). we adopt XGBoost [12] as the benchmark representative of machine learning approaches and train GNN-based model GraphMVP (46,, on the molecule editing datasct for comparison experimental results and setup details appear in Appendix E The results
26

---

indicate that XGBoost easily overfits to the training data, whereas GraphMVP exhibits significantly superior generalization compared to XGBoost, thereby validating the reliability of our proposed scoring approach.
C Further details on model implementation
C.1 Graph Tokenizer
Molecule to graph conversion. GNN is widely used in many scenarios, such as traffic 775 social relationships [76), and also molecules 61 Following the vanilla setting, we utilize RDKit experiments. For tasks involving single molecule as input, the molecule is converted directly. For tasks requiring multiple molecules as imput, only the first molecule in the imput sequence is converted into graph. Our model does not incorporate multi-gu multi--raph understanding, instead, it processes both the graph and SELFIES representation of the first molecule, while only the SELFIES representations are provided for the remaining molecules. Meanwhile, since MoleculeSTM ing compared to GraphMVP [46]. which leads improved multimodal model training efficiency, adopt MoleculeSTM as the graph encoder.
to transform SELFIES into graph structures
[45] incorporates additional molecular graph-text contrastive train-
Insertion. For graph tokens HG = {H, H2,-... Hn) after projection, we always insert the graph token at the beginning of user instruction X1. The input instruction will be updated to the concatenation of (Hg, X.
Multiple molecule inputs. In some tasks, (e.g., Reagent Prediction), when we need to copy with multiple molecules as the input, we stack them into batch dimensions and feed them to the graph encoder together, the resulting graph features are correspondingly concatenated together.
C.2 Multimodal Alignment
To balance the molecular graph and text modalities while ensuring training efficiency, we employ a single-layer linear projector in Stage 1. Following (45. carefully filter PubChem obtain 310K+ graph-text pairs and convert them into cnntructtoo-followiii data for pretraining. The alignment between the molecular graph and text modalities is enhanced solely by adjusting the parameters of the single-layer linear projector. After that, in the unified instruction tuning stage, we keep the projector active, allowing the projector to adapt to multiple tasks.
C.3
Gradient Adaptive LoRA (GAL)
The scaling factor is calculated as
70
+ B
(5)
rd
where 0 {a. p, }3}
Inspired by 30 53 23 we initialize these parameters as og = 16, 0.5, = 0. Additionally,
clip the range of these learnable parameters.
a = lap ao 00 + e) = clip(p. Po 8, po + 8) | = c3, Bo +
(6)
and we set 0.05, = 0.01. the rank set to 64, then 7y E [1.863, 2. 2.14]..
C.4 Mutual Representation Similarity
Task scaling setup. We build sequence of multi-task datasets with detailed composition as follows:
task:
Reagent Prediction.
2 tasks:
Reagent Prediction + Molecular Captioning.
4 tasks:
Reagent Prediction + Molecular Captioning + Solvent Prediction Catalyst Prediction.
8 tasks:
Reagent Prediction + Molecular Captioning + Solvent Prediction + Catalyst Prediction Forward Prediction + Retrosynthesis Property Prediction Yield Regression.
Similarity calculation. We first extract features BxLxxxx from decoder layers in L.M,, where B, L, T, dl is batch size, number of decoder layers, sequence length and the hidden dimension
27

---

Learning
Num Epoch
LR Decay
Stop Epoch
Batch Size
Warmup Ratio
Forward Reaction Prediction
Reagent Prediction
Retrosynthesis
Quantum Mechanics Property Prediction
Catalyst Prediction
Solvent Prediction
Yield Regression
15
10
Experimental Procedure Prediction
8o-5
cosine
128
0.0075
Description Q&A
Topological Polar Surface Area Prediction
Molecular Weight Prediction
Text Guided Molocule Generation
Molecule Editing
IUPAC2SELFIES
LogP Prediction
Molecular Captioning
10
Omni Molecular Tasks (2D & 3D)
15
15
Table 5: An overview of the parameersss and training configurations used in all molecular task experiments.
of LLM. The sequence dimension is then averaged.
R'= EE (R(43,43 4.) ml: 4)
Eiaml m|:,t)
where R' BxLxxd and RBxT is the mask indicating the padding tokens. We then flatten the first two dimensions and get R" (B+L)xd and calculate the similarity with mutual knn (27].
Let N B L, and we have two models A and B trained on different multi-tast datasets, we first find their & nearest neighbors knn and knn
knn KNN(RA, k) knn B = KNN(RR .k)
(8)
where knn" E IR N we then create indicator matrices
1.
j E knn [i.
1,
j E kmn 8 [i
M 1.
M, = 0,
i ajE1,....
N
(9)
0,
otherwise
otherwise
The accuracy of sample is
N Σ
acc(1]
|nnn [1,:] [14:]kknn knn
[i,n
8 [i,:]]
M M
(10)
j=1
Finally, the alignment score of two models is
Score Σ acc|1]
(11)
D Further details on training
Specialist models are typically fine-tuned on a single task at a time, repeating the process separately for each task, strategy known separate tuning. In contrast, generalist models undergo simultancous fine-tuning across multiple tasks, a process referred to as unified tuning. In this section, we present a detailed training framework for both of them in all experiments.
Separate instruction tuning. We follow training recipe outlined [0]. However, we observe significant overfitting when training the model on the molcap task for 20-50 epochs, as suggested in 10. To address this issue, we manually allocate 10% of the training set for validation and re- evaluated all tasks, we find that the recipes for forward prediction, reagent prediction, retrosynthesis,
28

---

and Quantum Mechanics Property Prediction from the original paper match our results, however, we identify an updated training strategy tailored the molcap task. The revised training recipe is summarized in Table
Unified instruction tuning, For unified training, we apply fixed training recipe shown in Table this recipe is consistent across all Unified Instruction Tuning.
For all experiments, the weight decay is set to O. The term Stop Epoch in Table 5 shows the epoch that the experiment stops. This is because the early stop mechanism we used to prevent overfitting.
E
Further details on experimental results
E.1
Baslines
In-Context Learning with DeepSeekV3 V3. To inspect the capability of In-Context Learning with powerful open-source model like DeepSeekV3, we randomly sample 5 examples from training set and feed the model with instruction, SELFIES representation and the answer. The model is asked to solve the task from test set with these in-context examples. The prompt template is as follows:
Instruction: Given the following examples: [Question: question Answer: answer * Answer the following question, Question: question.
here, question and answer are the specific samples from the dataset, *5 means we provide 5 question-answer pairs.
Molecular LLMs. Mol-Instruction, InstructMol, HIGHT, 3D-MoLM, and PRESTO represent a series of works leveraging large language models (LLMs) to perform molecular tasks. Among them, Mol- Iot-Innstuction, InstructMol, and HIGHT are specialist models that employ different adapters tailored specific tasks. 3D-MoLM offers both generalist and specialist variants, and we report the performance of both in Table PRESTO adopts a full fine-tuning strategy using single model. For datasets and tasks overlapping with ours, we directly report the results from the original paper. For tasks that PRESTO was not designed to handle, we conducted our own re-implementation.
Older Baselines. We include some older baselines such Llama2 (65] (result from
21),
nachO 449 (result from (II,, T5Chem (resul from (48)), TextChemTS [14 (result from 48] )
MolT5 19 (result from 48) ), Galatica (64 (result from 48 ) MolCA (result from 48
GPT-4-
0314 I (result from
10 BioMedGPT
52 (result from 10).
F.2 3D Adaptive Ability of Omni-Mol
Model
PPa
Esa
BLBU
Lev
RDK
MAC
Mur
Val
Model
APur
Esa
BLEU
Lev
RDK
MAC
Mor
Val
Forward Reaction Prodiction Task
Retrosythhsis
Omni-Mo
GL 2.28
0.73
0,980
5.55
0.895
0.477
0.87
1.00
Onni Mol
GL
2.28 0.57
0.960
8.97
0.864
0,909
0.83
1.00
Mol (3D)
0,983
(3D)
0.858
Model
Esa
BLBU
Mor
Val
RDK
MAC
Val
Reugent Prodiction Task
Cunlyst Prediction
Omni-Mal
2.38
0.23
0.726
0.52
Omni-Mol
2.28
0.947
1.00
Oumi-MMl (BD) GL.
28
0.25
0.706
14.35
0.603
0.663
0.56
1.00
Omni-Mot (SD)
GL.
2.28
0.77
0.336
1.55
0.913
0.898
0.78
1.00
Model
Type
4Param
110M0
LUMO
GAP
Avg.
Model
Type
4Pamm
B-2
B-4
R-1
R-2
R-L
Quantum Mechanies Propenty Prediction Yask
Molecular Captooning Task
Omni-Mool
GL.
2.28
0,0088
0.047
0.0049
0.0044
Omni-Mol
GL.
2.28
0.529
0.440
0.604
0.447
0.541
0.571
Omni (3D)
2.28
000366
0.0044
0.0037
Omni Mel (3D)
2.28
0.511
0.421
0.593
0.435
0.553
Model
4Phr
B-2
B-4
R-1
R-2
R-1.
M
Model
4Par
Weight
LogP
Description Q&A Tack
More No12Hfn
Omni-Moa
G1.
2.28
0.52
0.44
0.53
0.38
0.49
0.58
Ormi-Mol
GL.
2.28
11.07(100)
0.49(100)
Onzi Mal (3D)
GL
2:20
0.51
0.44
0.53
0.39
0.49
0.58
Orni Mel (3D)
GL
2.28
10.98(100)
0.48C100)
Levenshtcin
MACCS
Morgan
Validit
Solvent Prediction
Omni-MMl
0.52
0.759
0.671
B.673
1.00
Mal (3D)
GL
2:28
0.56
0.77
2.46
0.20
0:70
0.68
1.00
Table 6: Additional results of Omni-Mol with 3D GNN.
29

---

Model
Aec (Training)
Acc (Test)
ROC-AUC (Triining)
ROC-AUC (Test)
Molecule Ldtting
XGBoest
GraphMiv
0.8919
0.8744
0.94422
0.9188
Table 7: Comparison between XGBoost and GraphMVP on DRD2 score.
Medel
HOMO
LUMO
GAP
Weight
Log!
TPSA
GraphMVP
GraphMVP-G
0.3349
0.0076
0.0104
122.88
0.8099
42.33
GraphMVP-C
ST.
0.0061
0.0076
0.0104
123.57
0,7806
41.79
Omni-Mol
0.0038
0.0047
0,0019
11.07
0.49
5.89
Table 8: Comparison between Omni-Mol and GraphMVP
In this section, we answer the following question: Is it possible to develop a generalist 3D molecular LLM capable of effectively learning across diverse task domains? 3D molecules represent the physical form encountered in the real world, revealing richer layers of molecular information. Hence, enabling LLMMs achieve strong performance on 3D tasks critically important. more effectively encode 3D molecular graph data, we employ Uni-Mol [8] as the graph encoder, integrating it into our graph tokenizer. To obtain the 3D molecular graph information for the 3D Omni-Mol data, we follow the preprocessing pipeline outlined in Appendix B As the results shown in Table Omni-Mol achieves state-of-the-ar performance on all ten 3D tasks, demonstrating its strong adaptability to tasks that more closely reflect real-world scenarios and its potential significance for practical applications. For 3D unified training, we follow the hyperparameters and training strategies of the 2D Omni-Mol unified training. More details are provided in Appendix D
E.3 Comparison on Mo12Num Tasks with GNN
In this section, we re-implement GVaphMVP our baseline, we re-train GraphMVP on quantum mechanics property prediction, LogP prediction, molecular weight prediction, TPSA prediction tasks.
As shown in Table 8 Omni-Mol performs significantly better than traditional GNN models like GraphMVP. Omni-Mol provides up 91% improvement tasks like molecular weight prediction. GraphMVP-G introduces generative 2D self-supervised learning, i.e., it trains the GNN to reconstruct the attributes of nodes and edges that are randomly masked. GraphMVP-C introduces contrastive 2D colf-supervised learning, i.c., it trains the GNN to discriminate between constructed positive and negative molecular-get galph sample pairs.
E.4 Design Computational Tool for DRD2 Score
Since existing methods for computing the DRD2 score do not necessarily generalize to data with different distributions (55 71), in this session we design a new tool for computing the DRD2 score and demonstrate the superiority of the proposed approach through our experiments. For the Molecule Editing task dataset, we extract unique molecules and the DRD2 score labels provided by [7T]. To ensure that the DRD2 score computation method generalizes across different distributions, we enforce minimal similarity among molecules in the training, validation, and test sets, and maintain balanced active and inactive samples with the same threshold of activity in (55). Following (55), we employ the Butina clustering algorithm 8 yielding 50,552, 5,159, and 6,169 samples in the training. validation, and test sets, respectively. As stated in Appendix B we choose XGBoost and GraphMVP as representative non deepdeellea and deep-learning models, respectively. Adhering to the training setup of (55), the performance reported in Table 7 These results demonstrate that GraphMVP attains superior generalization on the Molecule Editing data, which underpins the reliability of DRD2 score evaluation for molecule editing tasks, since the generated molecules are unlikely have been encountered by the evaluation model during training.
30

---

Forward BLEU
Model
Exac
Levenshuein RDK
MACCS Morgan
Validiy
Model
Exact
BLEU
Levenshtein
RDK
MACCS
Morgan
Valdiiy
Reaosynthess
Head tuning 0.56
0.527
9.98
0.791
0,588
0.72
1,00
Head tuning
0.45
0.942
12,82
0.79
0.855
0.75
1,00
Levershiiin
Levansheiin RDK MACUS
Reagent Prediction Task
Caidyst Prodiction
Hend runing
0.662
17.74
1.00
Head nuning
0.71
0.727
2.46
Omni-Mal
0.25 0.7 726
14,59
0.557
0.627
0.52
1,00
Omni Mol
0.73
0.980
5.55
0.895
0.947
0.87
1.00
Model
BLEU-2
BLEU-4 ROUGE-1 ROUGE-2 ROUGELL METEOR
Model
Exact
BLBU Levenshtein RDK MACCS Morgan
Vilidiy
Molezilar Captioning
Solvent Prodiction
Head muning
0.450
0,358
0,546
0.359
0,477
0A822
Head tninn 0.39
0108
3.34
0.530
0.547
0.50
1.00
Onnii- Mol
0.529
0,460
0,604
0.541
0.571
Omni- Moli-MMol
0.759
0.671
0.64
Table 9: Additional ablation results on 6 tasks. We freeze the LLM decoder and only activate the
language model head.
F
More ablation study results
F.1 Ablation on Language Backbone
To ensure that the performance gain doesn't come from newer language backhones like Llama3. and Llama3.2, we replace the language model in Omni-Mol to Vicuna 7B [80] and conduct experience on several tasks. As shown in Table TT. with larger parameter size, Vicuna performs significantly better than Llama3.
F.2 Ablation on Parameter Efficient Tuning
One alternative tuning method tune only the language model head while keeping all parameters in the LLM decoder layers frozen. Note that for Llama 3.2 IB, the weights of the language model head are tied with the word embeddings; therefore, tuning the language model head also updates the word embeddings. The results are shown in Table where head tuning performs significantly worse than Omni-Mol.
F.3 Ablation on Clip in GAL
Clipping is employed to prevent the model from converging to local optima during training. To validate its effectiveness, we remove the clipping mechanism this experiment. As shown in Table 10 removing clip generally results in performance drop.
Method
Morgan
Levensticia
Validity
Casalyst Preduction
Orni Mol
0.742
0.794 0.755
0.11
0.899
0.747
1.843
1.000
wko
0.731
0.588
0.878
0.737
2.144
1.000
Forwad Prediction
Ocani Mol
0.885
0.941
0,863
6,05
1.000
wlo clip
0,703
0.980
0.365
0.927
0.840
7.045
1.000
Reapent Prodiction
Doni-Mal
0.266
0.749
0.586
0.651
0.542
14,026
1.000
wio
0,256
0.645
0.533
14,619
Reirosythesis
Doni-MMl
0.828
8.386
w/o clip
0.542
0.59
0.338
0.897
0.802
9.957
1.000
Orni Mul
0.554
0.782
0.703
0,700
0.678
2,498
1,000
0.550
692
0.667
2,550
Method
HOMO
LUMO
GAP
AVG
Method
B-2
B-4
R-1
R-2 R-L
Method
BH
SM
Quactum Mechanics
Molcap
Viaid
Ouni-MMll
0.0388
Omni--Mol
Omni-MMol
wio clip
00039
0.0055
0U0047
0UD047
wio clip
0.483
0.391
0.526
0.570
0.406 0.507
wño clip
0.958
0.680
Table TO: Ablation on clip in GAL
31

---

Model
RPar
Exa
BLBU
Lev
RDK
MAC
Mor
Val
Model
#Far
BLEU
Lev
RDK
MAC
Mor
Val
Renction Prediction
Proflcttion
Vicuna 78
50
St.
6,78
0.29
0.54
16.86
0,208
0.610
0.557
1.00
Vicuna 7B8
St. 6,78
0.29
0.62
4.21
0.454
0.430
0.392
1.00
1Jama3
1.38
1.00
LJama3
0.60
1.00
Medel
RL.F144
ROUGE:
ROUGE-2
ROUGA--
METFOR
Model
TogP
Melecular Captioning Task
More No12Nun
50
Llama3
16
[X
SL
1.38
0.455
03611
0.549
0.381
0.487
0,503
Chmaá IB
TB
SL
1.38
0.6690
Table 11: Comparison with different language backbone. We changed the backbonn to Vicuna 7B
and trained on several tasks, the results clearly indicate that better performance can be obtained from larger language model.
Catalyst Prediction
GT
Omai-MMel
Presto
Mo
Mu
Mn
X
N
Forward Prediction
Omai-MMl
Presto
NK
Reagent Prediction
Omni-Mol
Presto
Retrosynchesis
GT
Omai-Mol
Presto
?
?
Figure 7: Visualization of the cases generated by Omni-Mol and the baseline three reaction tasks.
G Case Study
G.1 Reaction Tasks
In this subsection, we visualize specific reactions in three reaction tasks. The results in Figure 7 and g demonstrate that our method exhibits more accurate generation capabilities compared to the baseline. For example, in the solvent prediction task, we are given the and as Omni-Mol
[C][C]]=C]C[[[Bracch VVCCCCCC//aaococcccccccoo/////// well as the product
reactants:
32

---

Case Milenie
Separate Tuning
Unified Tuning
The mellralo memberrt the dlsss of nendes that gyneole that arbeituue pasibore 1, and dic huubbred unreeeel (tril woameetyylaallla and respectirdy. it's metiool agrochent cal Bisondl. has xenobiotic mitabslito perecoles, dishkoroeesseee member seametly lenoeness otgantc sullide antrtle.
The no oi memberat the
The ol ammeber of the class gynai taat is lts perazole that aubetibstid it posoitin
clas of pyroolees the is
tyrool which is aststttued ot
and
1. sad by 2.4- and
dichlore.
deslore S-chomppeety influermmetsi safanyl respectordy. member
(iril uounelly byyeeryyllyylly.l.....
Hillummmedyl and grouge. respective;; dy, It's sibtile,
tyroolss, antrile duckereeeaate (iril Loumellylbeeee Mibrtbeeeeeeee
dichbmmeeaame. pennory
compound pynmiess aloabol,
andooc moc
member aePheremmyyl 2eteooooyyyyy/eeeee
Case Moleule
GT
Separate Tuning
Unified Tuning
The molauleis a stmudd and anion, the corjogat cabosyye anid. sllctic ne the 4a-4,5 5-saco pattway aemoli estrogm degniiiaion ot the boderium Syhungumone stain C.C. It conjugae leseof angeddieestooo- curboayle arid.
The monocarbecytic
te anj spale
syddunstmnne
upte 27 37-onoestadlll obtatned by depestonation of cadoayy group: nojor spactss ai congugate base at a ocoedtratil.
Ihalpha 17-egystrone. obranne by depeobaaiion of the
carbeay gruug mojor spedies
ge 7.8. it conrogate base
S-ucu- Saljjss F-egysaaonn
Figure 8: Visualization of the cases generated by unified tuning and separate tuning on the molecular captioning task.
Case 1: Mol
Case Inst. Pred
Case 1: Omni Pred
Property Info
Mol 1 QED: 8 1968
Inst. Pred QED: 2887
Omni. Pred QED: 0. 7783
Inst. Sim: 6765
Omni. Pred Sim: 4545
Case 2: Mol
Case 2: Inst. Pred
Case 2: Omni. Pred
Property Info
Mol 1 QED: 1898
Inst. Pred QED: 8 2445
Omni. Pred QED: 6.611
Inst. Pred Sim: 0. 3478
Omni. Pred Sim: 2119
Figure 9: Visualization of the cases generated by InstructMol and Omni-Mol on the molecule editing task.
correctly predicts the solvent as [CIN]]Brach)]]CCCCCC]])))) whereas PRESTO predicts an incor- rect solvent: [C][C][[N].
G.2 Molecular Captioning
In the case study of the molecular captioning task, as shown in Figure X the model's 's description of the same molecule becomes more accurate before and after mixed training. It is able to correctly classify learn identify functional
and localize functional groups. Does this suggest that the model groups from the reaction task? Additionally, constraints from other tasks the shared representation space also enhance the model's ability describe molecules. For example, for Case 1 Molecule, Separate Tuning outputs incorrect information regarding the locations of functional groups, whereas Unified Tuning predicts them correctly.
G.3 Molecule Editing
The case study of the molecule editing task is presented in Figure In this section, we analyze Omni-Mol's "modus operandi" in molecule editing from biological perspective, examining scaffold architecture, substituent patterns, and other insights to elucidate its potential real-world applicability.
We select optimized molecules meeting both unconstrained and constrained conditions and exceeding the QED threshold (0.6) for comparison, thereby revealing the optimization focus under each setting. And QED is modeled as the desirability profile of cight molecular properties, including molecular
33

---

Model
MW
ALOGP
HBA
HBI
PSA
ROTH
AROM
ALERTS
QLD Properies Case
Original
479.60
$1.18
Instrectblol
493.59
5.80
50.41 3
Omni Moi
397.52
3.80
54,04
QED Propetiess Case
2
Original
605.50
6.06
90.21
InsructMall
587.62
4.85
2
98.66
11
Omni Moi
434,54
4,27
67.26
Table 12: Comparison of QED Properties between InstructMol, and Omni-Man for two cases
weight (MW: 250-400), lipophilicity (AlogP: 1-4), hydrogen bond donors (HBD: 0-2), hydrogen bond acceptors (HBA: 3-6), polar surface area (PSA: 40-400000 90), number rotatable bonds (ROTB < 10), aromatic rings (AROM: 0-2), and structural alerts (ALERTS: 0) [5].
For case 1, both InstructMol and Omni-Mol retain the benzenediazonium core scaffold while targeting the wwwpeetooooppheeeeyaaammmmiiii
acid substituent, which features two hydrogen-bont donors, two acceptors, two rotatable bonds, and three structural deetss-mstt notably the carbamimi- dothioic acid (thiourea-like) moiety associated with acute toxicity. InstructMol merely replaces the terminal anisole with a 1.3-benzodioxole, failing to eliminate the toxicophore and producing negligi- ble changes in overall properties (only one rotatable bond is removed), thus yielding a minimal QED improvement. By contrast, Omni-Mol replaces the entire acid substituent with I-methylpiperazine, eliminating all alerts, removing aromatic ring, substan- tially reducing molecular weight (from 479.6 to 397.5 Da) and rotatable bonds, and lowering H-bond donors one. Benefiting from the hydrophilicity of the 1-methylpiperazine substituent, ALOGG is markedly decreased into the developable range.
baethoxyphenyJJearbbmiii
For case 2, under unconstrained conditions (no hard similarity requirement), the optimization is more flexible, better revealing Omni-Mol's s design logic versus InstructMol. InstructMol's molecule still contains multiple aromatic rings, whereas Omni-Mol removes excess aromatics, preserves only the core scaffold, and introduces small saturated rings (e.g., cyclopropyl) to enhance thrce-dimensionality and rigidity, mitigating risks of over aromatization. InstructMol's substituents include heavy-aton hydrophobic groups such as trifluoromethyl and fluoro on the aromatic ring, and a reactive aldehyde (-CHO) at the terminus. Omni-Mol, however, introduces unit and installs an umide-cyclopropang fragment, improving physicochemical balance and stability. For side-chain optimization, instead of InstructMol's 's long, highly polar chain, Omni-Mol cleverly replaces the bulky alkyl-alcohol side chain with a small, conformationally constrained cyclopropyl substituent, also boosting lipophilicity. These combined effects raise the QED score.
ethoxy group in place of the -CF3
In summary, Omni-Mol exhibits flexible yet systematic optimization strategy-scaffold tuning, side- chain simplification, and substituent "play to strengths and avoid weaknesses", achieving moderate contrast, InstructMol tends toward rigid expansion and structural complexity, lacking an adaptable. cohesive design rationale. Thus, Omni-Mol QED gains derive from rational, streamlined principles that balance flexibility and systemacity, aligning with medicinal chemistry guidelines and reflecting design ethos likely to resonate with real-world biochemical researchers, offering substantial practical value and significance. More detailed information about the QED properties be found in Table 12
molecular size and balanced hydrophobicity.
H task definition and prompt templates
H.1 Base Chat Template
For LLaMA 3.2 and LLAMA 3.1 instruction-tuned L.Mss, we use the base chat template suggested by the official documents, the multi-modal graph tokens are inserted at the beginning of user instructions.
34

---

System Prompt
<lbegin_off [enggt_of_text]_ Istart, _eeader_ id) bsystemes] end header_id]] > \aaaa
chat between curious user and an artilicial
intelligence assistant. The ussistant helpful, detailed, and polite answers the user's questions. [eot_id|>
Input
<lstart, header header_idd busers htt. istante lend_ header _idl \n
Assistant Output
Response Dosponse-llet__idl.
We use hewphntttue_rrgh_pppepp_pee _nladddd>>>> as pad token for SFT.
H.2 Forward Reaction Prediction
The forward reaction prediction task focuses on determining the chemical product of a reaction given its reactants and reagents. The forward reaction prediction task involves predicting the chemical product a reaction given the reactants and reagents as input. The input format is structured as the SELFIES representation of reactants, concatenated with a period (".") and the reagent information (e.g., "reactant irnnttanttr.reatttt "rractan22.raggent" The task requires the model to process this input and output the corresponding reaction product. The objective is accurately map the input reaction components their chemical outcome, leveraging the model's understanding of reaction patterns and transformations. key challenge in forward reaction prediction capturing the underlying chemical rules that govern reactivity. The model must infer how functional groups interact, recognize the role of reagents, and apply appropriate transformations to generate the correct product. This process requires deep understanding of reaction mechanisms, beyond simple pattern recognition. The prompt template is as follows.
Template
User Input: identifier>> @ @graph_tokken>/ Instruction. <leot_id|>> @ caasiatant, identifier>
GBELPIBS_eeactantr) CSELFIRS
Assistant Output: 4BLFIEE.producc>. product>.<leett
suzer suser_identifen identifier>: <lstart_ astart_headdr______ header [user< / aaddleedddeeeee headheedddrrrr j
<assistant idtrrratttdddtiiiir.>.. #lstart_headee Dassistani cnd__eaderr_____ headdoriidd>dpptp
Example
Instruction: reactants reagents provided, possible product
<SELFIES_ reagenta>: [Br][C[[][[r]] CCCCCC/CBBBBaICCCCC//CCCCCCCCCCC
[[Brraihh [[#Branch]][B]
reactttseeactttt:> [a+1]][OO111I]
<EELFTBs_product>: httpp//pwpppppooooooooooooooooto
H.3 Retrosynthesis
The retrosynthesis task focuses on predicting the reactants required to synthesize a given chemical product, fundamental challenge in organic chemistry and computational drug discovery. Unlike forward reaction prediction, which maps reactants to products, retrosynthesis operates in reverse, seeks determine the most plausible set of precursors that could yield the target compound under appropriate reaction conditions. This task is crucial for designing efficient synthetic routes, enabling chemists explore viable pathways for molecule construction while minimizing cost and complexity. At the core of this task is structured input format using SELFIES representations, ensuring a robust and unambiguous encoding of molecular structures. The input consists of the SELFIES representation of the target product, which the model then processes to generate the corresponding reactants. This structured formulation ensures that the model can generalize across diverse chemical transformations, learning the intricate patterns of bond formation and cleavage. A key challenge in retrosynthesis prediction handling the inherent one-to-many nature the problem: a single product can often be synthesized through multiple distinct reaction pathways. The model must effectively navigate
35

---

this complexity, identifying the most chemically plausible set of reactants based on learned reaction mechanisms. The prompt template is as follows.
Template
User Inpul: cuser_idents sea___ddntiieerr @ <graph_tokent gassistant_idantiftie
Instruction.
BEELPEE_pproduee
<leot_id|> d
Assistant Output: <SELFIES_ peeactattctttaaeee (eannttx</ooo___e [eot_id]>>>
intnidieetiierr slatart_eedder >user< lend header idl>>/nia
<assistant_ iieiaattdddentiieere.. <latart_ header, id Passisiants end beader_ _idderr_dddl idd>>lna
Example
Instruction: Which reactants could have been used to generate the given product?? The product is:
<SEFFIB (TT.A//ASSSSSSS//So/S//////////S
[BBanchh2]
#SBLFTBS_reactanee
H.4
Reagent Prediction
The reagent prediction task focuses on identifying the necessary reagents for given chemical reaction, a critical step in reaction planning and synthetic chemistry. This task is essential for guiding experimental chemists, as choosing the correct reagents influences reaction efficiency, selectivity, and feasibility. ensure a structured and standardized input format, we represent the reaction equation using SELFIES, robust molecular encoding system. The input consists of the SELFIES representations of the reactants, concatenated with reaction separator ">", followed by the SELFIES representation of the product. This format (e.g., "reactant .reactamt2produuet)) provides clear, machine-reade structure that allows the model to infer the necessary reagents based on known reaction mechanisms and transformation rules. One of the core challenges in reagent prediction is handling the diversity of chemical transformations. Different reactions require specific reagents that dictate the reaction type, whether it's oxidation, reduction, coupling, substitution reaction. The model must learn to recognize reaction context, interpret functional group interactions, and infer the most likely reagents from training data. The prompt template is follows.
Template
Inpul: caser, reagente><(eot
@ <graph_ n Instruction. @ <SELPIES, SELLP1ES_reactantts -SBLF1BS_producee (b
<lect_id|>> @ cassistant, identifier>
Assistant Output: <SELFIES +SELFIES_reagee cewrreeseeeeetteeee
iueer_ienttffier>: setart_beadess seddidduuueee >userr / hwwpheaadddrd___eeeeee
<assistant_ iidtttddddnnnnreeeee <lstart_heaade Dassistant< cnd__eader sed__haddrridd_pppp iid
Example
Instruction: Can you provide potential reugents for the following chemical reaction? The reaction
<SELFT6S_reactte
[OOJO[CCIICCCC]]]]j]nnnj]j]l] CUJ/////AAaaak//////o/////////h0 <SELFIES perdTtss_roodct>
(CC00//0hhhiiioo///////////////0 |ttp:////AAAAa////////////////// |OOor//cccc...iiii////////aAAIoa [[]]
<SELFIES <SELFIEs_raaeenta>> [CIC[[Baae//JICCCCOOOO//___-__
H.S
Molecular Captioning
The molecular captioning (Molcap) task focuses on generating descriptive textual information for given chemical compound based on its molecular structure. This task plays a crucial role in chemical informatics, enabling automated annotation of molecular properties, classification, and functional characteristics. MolCap leverages machine learning models to infer and generate human-readable descriptions that encapsulate key chemical attributes. The input for this task follows structured format using SELFIES, robust molecular representation designed for machine learning applications.
36

---

The SELFIES encoding of given compound serves as the input, and the model is responsible for producing descriptive caption that includes relevant chemical properties. These descriptions can encompass a wide range of molecular characteristics, such as compound classification (e.g.. "organic acid," "aminn-containing molecule"), pH estimation, presence of functional groups (e.g. "contains hydroxyl and ketone group"), solubility, toxicity, or other key features. One of the key challenges in molecular captioning is ensuring that the generated text is both chemically accurate and contextually informative. The model must learn to recognize molecular substructures, infer meaningful chemical attributes, and articulate these a clear and interpretable manner. The prompt template as follows.
Template
User Iaput: <user identifier> () token \n seeelitttat_ddttiiier""
Instraction.
<SELFIRS, <SELFIBS_compoun conpounds </eot_id|>> (
Assistant Output: Descriptio.< /eot__id_
<user <user_identifiee identifier>: astart_header sidddd//uueerr [usee<s and header hoaderridd>>ajj idi>)/n
<assistant_ idintantt_dd <latart_header searrt__eeer___ee Dassistants end htw__eeadde__rree _id
Example
Instruction: Please details about molecule. The compound SELFIES sequence
<SELFIE3_ cospound>][C]
=CC -|Bancch Braanchh/CC]C]]]ccpep BrancCC || BBannch])- [C]]Branch] [PPiC| [[ Biaa B|CC lolliiiiii
Description: The molecule organophosphate oxcanion obtained deprotonation phosphate groups (15-O-phospho--ett----ea baattt--iioofurnmssyyp...lo major spocies 7.3. It derives from D-ribofuranoss 5-phosphate(2-)) It is a conjugate base of a 1-5-0-ploopho--rst wtw.-b/aaeDiiloouuaoooliiiiiooio
H.6 Quantum Mechanics Property Prediction
The quantum mechanics property prediction task focuses on determining key quantum quantum-mechanice. mechanical properties of given chemical compound, providing critical insights into its electronic behavior, stability, and potential applications. This task is essential computational chemistry, materials science, and drug discovery, where quantum properties influence molecular interactions, reactivity, and optoelectronic performance. The input follows a structured format using SELFIES, robust molecular representation optimized for machine learning applications. Given the SELFIES encoding of molecule, the model is tasked with predicting its quantum properties, such as the highest occupied molecular orbital (HOMO) energy, lowest unoccupied molecular orbital (LUMO) energy, and the HOMO-LUMO gap. These properties are fundamental in determining a molecule's electronic structure, with implications for charge transfer, chemical reactiviry, and photophysical behavior. One of the key challenges in quantum property prediction is capturing the underlying centuum-chemical interactions that govern molecular behavior. The prompt template is as follows.
Template
User Input: <user identifier> <graph_tokon)) Insiruction. caesistant_idees iddetiiieee
<BELPIBS_ comppuudd <lest_id|>> +
Assistant Output: P eat_id|>
<user <user_identiff Eettarttheader_ttt seaerrdd//useee Duser< and header_iddp)aml idj>)nnin
<assistant, beeeistantt_deniifiiee <latart, beader, id Dassistant< ledded__dddde beader, IdI>\nn
Example
Instruction: What the HOMO-LUMO of this molecule? The compound SELFIES sequence is:
<SELFIBS_ compound>: <:ELPIBs_compound: ----CC/OOOOCCCCCCCCC----------- (aramccll/RRggg" [Kngg]]=NN|/Ring [|=Branch]
Property: 0.1487
37

---

H.7
Catalyst Prediction
The catalyst prediction task focuses on identifying the appropriate catalysts required to facilitate given chemical reaction. Catalysts play a crucial role in modifying reaction pathways, lowering activation energy, and improving reaction efficiency without being consumed in the process. The input follows the SELFIES representation, robust molecular encoding system designed for computational applications. The reaction is expressed as an equation where the SELFIES representations of the reac- tants are concatenated and separated from the product using ">" (e.g.. "reactant Ireactaatl.raacta22222231 reactant2aproduet)) This structured representation allows the model to process the reaction as whole and infer the most suitable catalyst that enables the transformation. One of the primary challenges in catalyst prediction is understanding the nuanced role that catalysts play in different reaction mechanisms. Unlike reagents, which directly participate in the reaction, catalysts provide alternative pathways to enhance reaction kinetics. The prompt template is as follows.
Template
User Input: iddntiiiir>> @ <graph_ 1orapph_t Instruction.
<SELFIES_produsee (
<leot_id|>> @ cassiatant, identifieett
Assistant Output: <|atart_
<user <user_ident sarr__dddtiifirr:
ceettart_eeeerr.... header Dusser aaddenn_addll headeeadddr__ htt_hhadddd_ddddaall _1d/>\nnnn identifiers: ieriistttt_ddetiifieeeeee leeaet, Id bassistants bed__bederr_ jesdde_
<aseistant,
Example
Instruction: Given this chemical reaction. what are some catalysts that could have been used? The reaction is <SELIIBS_racttatter reattantss:::: [C][C]]=C]]]]]][[ Baal::i Branchl [CC[N][[]]CC[[iinn [BBBB (iircci222yIBBaaaaahiC/HHHHBBB ch]][C][[][F] c B[CCCCC[--Banacc]]]]]]]-aa----
[[=Branchl [[r]]
<SELFI63_ #SELFTBS_product>: [N][[[]C[[[C][]]]][[[[aaaa [[Ring [[C][[[[[[[[[]]]]] B[ccC//[/BaachiBBaaaaaaaaaaaap [[c [Boosah][C[F
<SELFIE3_ hww.rrss_catalyaaa:/0000 htt.://www..................///0 hhhp0/0000000000r000/0/0hCoeeohh
H.8
Solvent Prediction
The solvent prediction task focuses identifying the appropriate solvents required for given chemical reaction. Solvents play crucial role in determining reaction efficiency, influencing factors such solubility, reaction kinetics, selectivity, and stability of intermediates. To ensure a structured and machine-readable representation, the input follows the SELFIES format, robust molecular encoding system designed for computational applications. The reaction expressed as an equation where the SELFIES representations of the reactants are concatenated and separated from the product using the reaction separator ">" (e.g., "reactant] .reactant2sproduut"" This structured format allows the model to interpret the reaction context and infer the most suitable solvents required to facilitate the transformation. One of the key challenges solvent prediction is understanding the diverse roles solvents play in different reaction mechanisms. The prompt template is as follows.
38

---

Template
User Input: cuser, identifier> @ 'graph_tokent token> Instruction. @
<SBLFIBS_ @
<lest_id|>> @ cansistant, 1
Assistant Output: (ocGLFFEB_oolveta>><(/1
suser duser_identifie>:: aatart_headdrr....... <uueerc/ andd header_iiddaa 1n
Sasaistant_1dentiTiens <latart_ [latart_header. Passistants end__ed__dddd _tt__er_dddddaaaaee_e .idn
Example
Instruction: Plense propose potential solvents that might have been utilized in the provided chemical reaction. The reaction
Is
[Branch]]
<SELFI68_pPF BBELFIEs_product:
[Ring]][N]
35ELF163_8o1vent [0]
H.9 Yield Regression
The yield regression task focuses estimating the proportion of the actual product obtained in chemical reaction relative to its theoretical maximum. Reaction yield is a critical metric in organic synthesis, pharmaceutical manufacturing, and industrial chemistry, as directly influences process efficiency, resource utilization, and cost-effectiveness. The input follows the SELFIES format, robust molecular encoding system tailored for computational chemistry. The reaction expressed as an equation where the SELFIES representations of the reactants are concatenated and separated from the product using the reaction separator ">" (e.g., "reactant .recctant2 sprouuctt"""" This structured format provides standardized input for the model, allowing it to interpret the reaction context and estimate the expected yield. One of the key challenges in yield prediction capturing the complex interplay between reaction conditions, molecular stability, steric effects, and solvent or catalyst influences. The prompt template is as follows.
Template
User Input: Kuser, ifier @ <graph,token>ly
token>) @ Instruction. @ Assistant Output: Shoperty<</eat cot_idl>>>
<SELFIES, 1SEFFIES_product product> c
<lect_id|> @ bbaiistant_identiie
<user_idens iidenttttrrr: aetart__hedder.. beader_id//ussr<// header__ddddaa (nj
<aeeistant, identiifeede <latart_ beader, id Dassistant< beadere Beraadr__ddl>lp idi>)nnn
Example
Instruction: Please propose potential solvents that might have been utilized in the provided chemical reaction. The reaction
is
<SELFIB9_reectene
[|CpC//cC/CCCCCCCCCCCCCCCCCCCC [ [[
[[i] || 110 |BBBBBaal [[[]|]]]]]]]]]]]]] Branch2jjnggg ||=Branchi [[O]-
[BI[N]C[]==CCC]]=CCC]]]==---] [[] [|Braall]]]hhhhh]eeCeC [|C-:a//]]][[CCCCCC]]]CCCCCCC--- [[]g:] [[Ring] [ [[C]
[CccI]][[[[[FFCC]]]]]]]C [[]]]][[[Rmmnhh]]]]]]]
[CccCCCC[[[chh[[Rnnnggggg [[[]a][[[[Baacce]][[[e [[Branch]]
[aaaachhh||innn][[[[[[i.. |Rnge[[[Riinn|B/BBBanneea Br [[[Riing][[BBande]] Branch2][[[[RR [[+Brunch]][C)-
CCICICCCICC[BBBBccCHCCCCCCCCCCCC
[JCCCICC]J]OOCCCC[CBBBCCc [[C]gg]]C[][Ringg |||=Brucch]]|Cllee [|=Branch2]] [CI]iig [|*Branch2|| [[=
[|=Branch]] [C]]]//h[[[]]BBrccchhh |CCC[///ccccccCC]CCCCCCCCCCCCCCC
[[C]]] [[C][[]]]]
CCC.IIICCCCCCCCCCCCCCCCCCCCCCCCC
[iBanccii// BBBracchh
[[ [|]ra]Chl [[C[[]][]]]]]]]] [[=Bracch]]
#BBLFTes_product> [C]CC]C[[[]]]C]]Baa Bra]cC]]Banchh2]]Riii |[R
(CIF[[Breehh]C]]]]]PII [JCIFIFICC [[+BBacch2[C]]]-[[iinnl]l
Property: 0.14
H.10
LogP Prediction
The LogP prediction task focuses on determining the octanol-water partition coefficient (LogP) of given chemical compound, key physicochemical property that influences molecular behavior
39

---

across various environments, LogP quantifies the relative solubility of a compound in octanol versus water, serving as critical indicator of lipophilicity, hydrophobicity, and membrane permeability, The task employs the SELFIES molecular representation, which encodes chemical structures in machine-readable form optimized for deep learning models. Given the SELFIES representation of a compound, the model is responsible for predicting its LogP value, numerical measure that typically ranges from negative values (indicating high water solubility) to positive values (indicating high lipophilicity). This structured approach allows the model to learn patterns between molecular structure and partitioning behavior, enabling accurate and data-driven LogP estimation. One of the key challenges in LogP prediction is capturing the complex molecular interactions that dictate solubility behavior. The prompt template is as follows.
Template
User Input: <ueer identifier> "graph_ @ Instruction. sussittant_identiiier
@BLFIES_compouue <leot_id|>> +
Assistant Output: eat_idd
<user sarr__dddntifirrr <latart, header, Lid (ussersrreaan header header__dd>)@ppp idi>)nnn
<assistant identifiert Basissaant_dentiffiez
cistar__heades Passistani< hand_eeddrr___ paaador_idd>>p i
Example
interested LogP
you
uncertain, provide an
Respond with the numerical value only. The molecule SELFIES sequence is:
<SELFIES, compound>: C[[[][C[==[[CC]]=BBaaa [| [Beaacch][[CCI[]II-BBB JRing]]][RiR If= [[[O HI [| BrBnnch] /|cpecchhl/CIOOOooo [|Cin]] [ranchl|]CCr e [|anchh) [|=Branchl [[O]|C]]Ring Arann--
[b][=o[[C[==C]|]]]BBBaaaa [[= [[]aach])nc)) [[ [[]eeee[[[[BBBrahhe [[Rinn]][[C]][eee B|
Property: The Logl' for the inpul molecule is 2.00.
H.11 Molecular Weight Prediction
The molecular weight prediction task focuses determining the molecular weight a given chemical compound, a fundamental property that reflects its size and atomic composition. Molecular weight is crucial parameter in various scientific disciplines, including organic synthesis, drug design, polymer chemistry, and materials science. It influences key aspects such as reaction stoichiometry, diffusion rates, bioavailability, and stability. The input follows the SELFIES format, robust molecular encoding system designed for computational chemistry applications. The input consists of the SELFIES representation of molecule, which the model processes predict its molecular weight in unified atomic mass units (Da). This structured approach allows the model to learn the relationships between molecular structure and atomic composition, enabling precise and efficient molecular weight estimation. The prompt template is as follows.
Template
Inpul: identifier> <graph_token>>
<S8LPIBS_compound
<leot_id|>>
<assistant, idantifier>
bassistant_tdettiffee>>
Assistant Output: Ploperty<<l 1d
<ueer. ideetidddetiier> <latart_beaade beader, >users end, head_header_ddd id/>)
cassistant, iaiittaat_idettiierr""" llstart_beader, bussistant| end_er aeeaderrrddlleeeeeeeeeeeeeeeeeee peldddrriddlllp idi>)pp
Example
Instruction: Please provide me with the Molecular Weight value of this molecule. Determine the Molecular Weight value this molecule. uncertain, provide estimate. Respond with the numerical value only. The malecule SEL FIES sequence is:
compoundd: (TOOIICI--CIBaaancciij/aaaaee [[C= [[][[[[ii]][aaaaaaaall [[CC [[C- [CIC|ICCICII[][Beee// [Branch [[]]n]] Br-nBBBccBh [SSaancch]][[[BBnddda [[[[[[[[[[[[[[[ma]aa]]] [[C[ (CIIOOINNICCCCCCCCICIICIIIIIIIII Beaaache]]C]]]]]]]]]]] BBannch)
Property: The Molecular Weight the input molecule 491.60 g/mol.
40

---

H.12 Topological Polar Surface Area Prediction
The topological polar surface area (TPSA) prediction task focuses determining the TPSA value of a given chemical compound, a key deseriptor that reflects its molecular polarity and hydrogen bonding capacity. TPSA is widely used in cheminformatics, particularly in drug discovery, where it serves as an important predictor of solubility, permeability, and absorption. compound's TPSA value influences its bioavailability, blood-brain barrier penetration, and interactions with biological membranes, making accurate prediction essential for pharmaceutical and materials research. The input is the SELFIES representation of the compound, and the model is tasked with predicting the compound's 's TPSA. The objective is to provide insights into the compound's 's polarity, solubility, and potential absorption characteristics, which are crucial considerations in areas such as drug discovery and materials research, The prompt template is as follows.
Template
User Input: <usar iddntifier>> Tgkanh_ ieeeistant_idettiffne
@ Instraction.
<SELFIES_ compound> </ect_id|> ()
Assistant Output: Poopett.<]
<user <user_identife Lddntifie>>:: sttart_headdr, (usssrrreeea header _il>>/ end meader id
<aeeistant, "aeiistant_ieettiffeee <lstart_ beader,,
Example
Instruction: would know Topological
molecule, can you provide uncertain,
provide am estimate. Respond with the numerical value only. The compound SELFIES sequence is:
<SELFIES_
httpress_ooooodd>>>__ compound>:[[]]e. Bra]C[[[BBncch]]jaall [|
[[O][[][[Ranch [[O]]] [[Rnanch][[inn]]]]]]e
ngl][=Branch| [[BBaaccc][[CCCCCCCCCCCCCCCCCCCC [[CCCCC[CCCCCC[[BcBacCaCCCCCC ngl]|=Branchl [| (iagel/RRll///aaaaa |||=Branch]]] [[]]cch]]OO[[]]]Riiee ||Riia]||BBrachaaa [[HBrachh2]]]] [[]]]|Rng2]]|Rinn I|C [|C][[]]]
Branch2[[Ri-
gl][Ring|][O CINCCCCCCCBBIIIIIINIINIIIICCCCCC
[][ [1-Branc2][[inng [[[=Branch] [[[=BracchJjCCCCIIC
Property: The Topological Polar Surface Area for the input molecule 96.50 As.
H.13 Description Q&A
The description question and answer task involves responding to queries regarding a given compound's properties in physical chemistry and related fields. The input is the SELFIES representation of the compound. The model is tasked with providing accurate the compound's physical and chemical attributes, encompassing a broad range of topics-from pharmacological considerations to the specific influence of structural and functional groups on biological activity (e.g., in anticancer agents such as 4- Hyydroaycyclophoshhhmde or Lobaplatin). The objective achieve comprehensive and in-depth understanding of the compound's characteristics. The prompt template is follows.
detailed questions about
Template
User Input: <user seser_identiffer? <graph_tokeng token>> @ Instruction. srreiittatt_ienttrree
<BELFIES_ comtpsudd <leot__id>> @
Assistant Output: inscrrption./eett.....
<user. idantifier:: aettart_headr_____ Dusser and_land_daaaaa headereadrrrr. In'>)
1eettatt_deettiierr" "istart_header. bassistant< end_ header. (idlderrddd>>>ppp
Example
Instruction: What is the main component of Lobaplatin that gives its anticancer properties? The compound SELFIES
sequence
<SELFIES compound>: [C[cc @ [[Branch [][ Branhh Bene Biccc IIC-:/--OOO------1I/C----------- /CCp:///CCCCC///////CCCCCCCCCCCC
[=Branchi][C@
Description: main component Lobaplatin that gives anticancer properties is the platinum cation (Pt+2).
41

---

H.14 Experimental Procedure Prediction
The experimental procedure prediction task focuses on generating detailed, step-by-step protocol for conducting chemical experiments based on given set of experimental components. This task plays crucial role in automated synthesis planning, tightthroughhut experimentation, and robotic chemistry, where structured experimental procedures are essential for reproducibility, efficiency, and accuracy. The input is a mapping between experimental components and their corresponding SELFIES representations (e.g.. "Reactants: Sindex$: SELFIES" ...). The model is tasked with producing a structured sequence of operations that associates each component with the detailed steps of the experiment. The objective is to automate the chemical synthesis process by providing exccutable, structured experimental procedures. The prompt template is as follows.
Template
User Input: cuser, identifier> @ Sgraph, tokehtttke-uuuuu (u @ Instruction. @ <IDX_ Reactants MAP> MAP>> @ CIDX_Product,,oo MecP $ <leot_id|>> @ <assistant, iddnttifier Assistant Output: <ACTION_ <ACTICM_Sequennee setuancc><//oot______.._ id]_
Catalysta, @ SIDX_Solvents_Wee
<user_identing < llatart__hadder busee<< Swader__id>\lp
ieeeiitant_identiiee
aestar_beaddes id Passistant< lndd beader. idi>)/nn
Example
Instruction: From the provided starting materials and larget compound, generate step-by-step experimental protocol.
The Action Sequence:
[Ring] [BB//Brach]]]CI [[C[]=CC[[RRRgg]]=CCCEED SELFIES S3$: (START SAASSSSSSSSSSAAAALALLASLSSSSSSIC SUU.K//.haaaaaiiii/ll/aaalllllla
[J[K+IJ[END_
SEIIINDDSELLFIISS SELFIES]
<IDX_ Product_ www.Prrct.APPPPPoouuuuee
1MP==PProdattS-11:/STAAK SEI http://wwww....ooooo.o..ocoo///a
SELFIES]
ranchl
1|JEND_SEE SELFEES
<IDX_ S0Xy_Sslvent_MAle Solvents: $2$; [START_ SAAALLLEEELTEESSSCCCCOIIIIIIIEEL
ACTION Sequence: ADD SIS (2 liter, 57.2 ADD S2S (500 ml) ADD $25 STIR: ADD $3S (18.0 ADD $45 g): FILTER; WASH
WASH sodium chloride WASH with with sodium chloride:
DRYSOLUTION over magnesium sulfate FILTER keep filtrate: CONCENTRATE; YIELD (20.8
H.15 IUPAC2SELFIES
In the IUPAC2SELFIES task, the model asked generate the SELFIES representation molecule based on its IUPAC name. The IUPAC (International Union of Pure and Applied Chemistry) name is standardized and systematic way to uniquely describe molecule's structure using rules for parent chains, functional groups, and substituents. To succeed in this task, the model must accurately understand these rules and produce the correct SELFIES representation accordingly.
Template
User Input: <user <user_identil iddntifier>> @9 Instruction. @ cassiatant, iiaisttaatt_ddeniifiar
Assistant Output: <SBLFIES, cSBLF12S_compant compound><(los
Kuser Kuser_idente identifier>: _latart_header., _1d Duserc and heaheeaddrrr header__id>)(nn _id|>\nnnn
clatart_beader id Dassistants end_beader, idi>>
Example
Instruction: What the molocular structure corresponding IUPAC name? The IUPAC name is: N-1--amino-2-
<SELFIES, compound>: htt.a//wwww....aaaaaaaaaaaaaattn
|Branchl hcccC///BBaacccCCCCCCCCCCCCCCI
42

---

H.16 Text Guided Molecule Generation
In this task, the model is asked to generate a molecule in SELFIES notation based on textual description. The description includes information such as functional group substitutions, structural features. and desired properties. The model must comprehend this complex information and design molecule that satisfies the given requirements.
Template
User Input: <user suser_identifie iddentifierr @ Instruction. @ "essistaat__ddntifiiie Assistant Output: cocpsssooomoddcceee poo__dd>>>
<user, serd_dddetiierr: <lstart, header_earr____ id Puser< and saad_hhader____e_ _idl>)nn
<assistant, identifier:: <latart_ id bassistant< htw_heedddddddYY/
Example
Instruction: Design a molocule according description provided. description molecule nitrile propionitrilc which one of the methyl hydrogens has been replaced by phenyl group. It is nitrile and member of benzenes. functionally relatedd to an acetonitrile. The molecall natural product found Brassica napus, Brassica oleravea, and other organisms with data available.
<SELFIBS_ compound>: [C]]=C]]([]=C(|]Braag Branchh |CCC:CCCCKKiiggggBBBaaaaaaaiiaia JCCCCCCCCC
H.17
Molecule Editing
The Molecule Editing task is concerned with optimizing one or more target properties of given molecule, particularly the DRD2 score and QED. Specifically, the objective is to apply structural edits to molecule with low DRD2 score and low QED in order to generate variant exhibiting higher DRD2 score and QED. The model is tasked with optimizing the QED DRD2 score an input SELFIES molecule while maintaining some extent of structural similarity. It outputs the SELFIES of the optimized molecule. The objective optimize specific properties of a given molecule so as to yield novel biological insights. The prompt template is as follows.
Template
User Input: cuser, seer_identifier> @ sccapb_token>> + Instruction. @ @SELFIES_compoune Inpuut> @ <leot_id|> + <aseistant_ideet iddatiiieee
Assistant Output: <SELFIES_ compoundd><(aos [
<user_ <user_identif fiar:: <latart__eeaee >users and hwwdeeeaddrdddeeeeaeeeeeee idd>)\
aastart_beadeee id Dassistant< med_beeder, fheader_idlle ini
Example
Instruction: Develop new molecular structure that cahances the QED score and koops structurel similarity the initial molecule. The compound SELFIES sequence is:
<SELFIES_ <SELPIBS_compound. Input>: /tt.:///CCcC//i/oooo//o////////e
<SELFIRS comppoudd_
I Discussion on Generalist and Specialist
Here, we define the notions of generalist and specialist models. A generalist model refers single model with shared parameters 0 that is trained across an entire set tasks T (T,). .. In contrast, a specialist model assigns distinct set of parameters 6, to each individual task T,.
Although the authors of InstructMol and HIGHT claim their models be generalist their respective papers, they in fact employ distinct LoRA adapters for different tasks, following the formulation 0: = Og + Ohoray where Og denotes the pretrained L.LM parameters and 0 represents the task- specific LoRA parameters for task T, As result, these models do not satisfy the criteria for being
43

---

Confguration
Exact
BLEU
Levessitein
KDK
MACCS
Morgan
Validity
Learning Perfeemance
Retrosy utheess
Renosyutheiis only
0.55
0,55
9.93
0.3
0.89
0,80
1.00
Retrosyafheiis
Table 13: Continual Learning performance relrosynthesis task before and after learning on forward prediction.
true generalist models. In contrast, both PRESTO and Omni-Mol utilize a single parameter set shared across all tasks.
Discussion on Continual Learning
Instead of unified instruction tuning, which trains the model to learn all tasks simultaneously in single procedure, is also plausible train the model sequentially on various tasks.
We conduct an experiment in a continual learning fashion: we first train the model on the retrosynthesis task and then the trained model learn forward prediction. We record the performance the retrosynthesis task both before and after training on forward prediction. As shown in Table [3] after training on forward prediction,
observed significant performance drop. This suggests that the model may forget previously learned knowledge and cannot benefit from earlier tasks. Therefore, in Omni-Mol, we train all tasks jointly.
44


Reviews:
[{"review_id": "nsWkcTK6FO", "reviewer": "Reviewer_fT5E", "summary": "This paper introduces Omni-Mol, a unified and scalable multimodal large language model tailored for molecular science, capable of handling a wide spectrum of tasks across four modalities: Mol2Mol, Mol2Text, Mol2Num, and Text2Mol. The framework leverages a novel Gradient Adaptive LoRA (GAL) mechanism and a large-scale multitask instruction dataset (>1.3M samples across 16 tasks) to enable effective parameter-efficient fine-tuning. The authors report strong performance on a variety of benchmark tasks, and their model surpasses several larger-scale baselines in terms of both effectiveness and efficiency.", "questions": "1. How sensitive is GAL’s performance to hyperparameter tuning?\n\n2. How well does the model generalize to truly novel molecular tasks not seen in training?\n\n3. In tasks requiring multiple molecules as input (e.g., reactions),  how graph encoder handle multi-input task?", "limitations": "1. Due to limited computational resources, the current study is unable to scale Omni-Mol to larger parameter sizes. This restricts the exploration of potential performance gains from model scaling or deeper expert routing.\n\n2. Although instruction tuning is central to the framework, many prompts are manually templated and task-specific. There is no mechanism for dynamic prompt generation or adaptation, potentially limiting model usability by non-expert chemists or in unstructured settings.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. The authors propose a novel and comprehensive multimodal LLM framework that supports four distinct categories of molecular tasks: Mol2Mol, Mol2Text, Mol2Num, and Text2Mol—this \"any-to-any\" modality design.\n\n2. The authors propose Gradient Adaptive LoRA (GAL), which dynamically adjusts low-rank update magnitudes based on task-specific intrinsic dimensions, thereby enhancing multi-task adaptability.\n\n3. The authors curated a dataset comprising over 1.3 million samples across 16 tasks, making it the largest molecular instruction tuning dataset to date.\n\n## Weaknesses\n1. Despite the claim of “unified instruction tuning,” the work lacks an empirical analysis of robustness to instruction variability. The paper does not report performance under paraphrased or noisy instructions, nor does it assess zero-shot generalization to unseen prompts.\n\n2. While Omni-Mol demonstrates strong results on most tasks, its performance on the Catalyst Prediction task lags behind several baselines. This suggests potential difficulty in modeling subtle structure-function relationships specific to catalysis, which may involve sparse and high-variance distributions.", "was_revised": false, "final_justification": "", "rebuttal": "> Q1: Lack of empirical analysis of robustness to instruction variability.\n\nA1: Thanks for the comment. In our dataset, each input instruction is paraphrased using at least six different prompt templates. During evaluation, we also test the model with various paraphrased prompts rather than relying on a single fixed instruction. You can refer to the repository link for the full set of prompt templates. In fact, each sample is paired with a uniquely crafted prompt, making the model robust to varying \"zero-shot prompts\". We would be happy to explore further directions and experiments if you have additional suggestions.\n\n> Q2: Difficulty in modeling subtle structure-function relationships specific to catalysis.\n\nA2: Thank you for the comment. Our Omni-Mol model has already demonstrated strong performance on 13 datasets. Although it slightly underperforms PRESTO on the catalyst task, it’s important to note that PRESTO benefits from two additional advantages: (1) it undergoes multiple stages of pretraining, including approximately 3 million samples of synthetic procedure descriptions and molecule name conversions, whereas the LLM backbone of Omni-Mol is used without any pretraining; and (2) PRESTO is based on a 7B-parameter model, while Omni-Mol uses a significantly smaller backbone of around 2B parameters, leaving substantial room for future improvement.\n\n> Q3: How sensitive is GAL’s performance to hyperparameter tuning? \n\nA3: Thanks for the comment. We tune the rank of GAL and the results are shown in the table below. Due to the time limit, we train the model on 8 tasks.\n\nCatalyst\n| Method    | Exact | BLEU  | RDK   | MACCS | Morgan | Leven | val   |\n|-----------|-------|-------|-------|-------|--------|-------|-------|\n| RANK=32   | 0.733 | 0.797 | 0.903 | 0.887 | 0.739  | 1.948 | 1.000 |\n| RANK=128  | 0.751 | 0.815 | 0.910 | 0.893 | 0.758  | 1.911 | 1.000 |\n\nForward\n| Method    | Exact | BLEU  | RDK   | MACCS | Morgan | Leven | val   |\n|-----------|-------|-------|-------|-------|--------|-------|-------|\n| RANK=32   | 0.686 | 0.978 | 0.864 | 0.928 | 0.840  | 7.458 | 1.000 |\n| RANK=128  | 0.783 | 0.986 | 0.906 | 0.952 | 0.888  | 4.714 | 1.000 |\n\nQuantum Mechanics Property Prediction\n| Method   | HOMO  | LUMO  | GAP   | AVG   |\n|----------|-------|-------|-------|-------|\n| RANK=32  | 0.0039| 0.0040| 0.0047| 0.0042|\n| RANK=128 | 0.0037| 0.0040| 0.0046| 0.0041|\n\nMolcap\n| Method   | BLEU-2 | BLEU-4 | METEOR | ROUGE-1 | ROUGE-2 | ROUGE-L |\n|----------|--------|--------|--------|---------|---------|---------|\n| RANK=32  | 0.477  | 0.384  | 0.531  | 0.571   | 0.406   | 0.508   |\n| RANK=128 | 0.515  | 0.426  | 0.562  | 0.596   | 0.438   | 0.533   |\n\nReagent\n| Method   | Exact | BLEU  | RDK   | MACCS | Morgan | Leven   | val   |\n|----------|-------|-------|-------|-------|--------|---------|-------|\n| RANK=32  | 0.200 | 0.679 | 0.509 | 0.588 | 0.472  | 16.711  | 1.000 |\n| RANK=128 | 0.283 | 0.749 | 0.591 | 0.655 | 0.557  | 13.624  | 1.000 |\n\nRetrosynthesis\n| Method   | Exact | BLEU  | RDK   | MACCS | Morgan | Leven  | val   |\n|----------|-------|-------|-------|-------|--------|--------|-------|\n| RANK=32  | 0.525 | 0.955 | 0.827 | 0.891 | 0.789  | 10.389 | 1.000 |\n| RANK=128 | 0.593 | 0.961 | 0.860 | 0.911 | 0.832  | 8.343  | 1.000 |\n\nSolvent\n| Method   | Exact | BLEU  | RDK   | MACCS | Morgan | Leven | val   |\n|----------|-------|-------|-------|-------|--------|-------|-------|\n| RANK=32  | 0.461 | 0.718 | 0.603 | 0.612 | 0.573  | 3.037 | 1.000 |\n| RANK=128 | 0.588 | 0.795 | 0.730 | 0.726 | 0.706  | 2.295 | 1.000 |\n\nYield\n| Method   | BH     | SM     |\n|----------|--------|--------|\n| RANK=32  | 0.8877 | 0.6114 |\n| RANK=128 | 0.9430 | 0.6674 |\n\nIn the experimental results, we observed that an increase in rank leads to performance improvements, validating the robustness of our method. The growth effect are not particularly evident in Quantum Mechanics Property Prediction task, but significant increases were observed in other tasks.\n\n> Q4: How well does the model generalize to truly novel molecular tasks not seen in training?\n\nA4: Thanks for the comment. Generalizing to out-of-domain data is indeed an interesting aspect. Coincidentally, one of the reviewers pointed out that we did not fully consider the molecular graph classification task. This gives us a great opportunity to use it as a testbed to evaluate the generalization capability of our model. We directly implement the zero-shot evaluation on three different classification datasets [1, 2]. Since the original pretrained weights of many baseline models are not publicly available, we use Qwen3, which represents the state-of-the-art on scientific tasks, as our primary comparison.\n\nBBBP\n| Model        | F1 Score            | Accuracy   |\n|--------------|---------------------|------------|\n| Omni-Mol      | 0.5725  | 0.4363     |\n| Qwen3      | 0.0617 | 0.23039    |\n\nHIV\n| Model       | F1 Score            | Accuracy   |\n|-------------|---------------------|------------|\n| Omni-Mol     | 0.0254 | 0.0650     |\n| Qwen3  | 0.000  | 0.03383    |\n\nSIDER\n| Model             | F1 Score            | Accuracy   |\n|------------|---------------------|------------|\n| Omni-Mol   | 0.4054 | 0.3846     |\n| Qwen3 | 0.1553  | 0.4891     |\n\nAs the experimental results show, Omni-Mol demonstrates certain performance on completely unseen classification tasks, slightly outperforming Qwen3 in some tasks and metrics.\n\n> Q5: How graph encoder handle multi-input task?\n\nA5: Thanks for the comment. When multiple molecules involved, we stack them into batch dimensions and feed them to the graph encoder together, the resulting graph features are correspondingly concatenated together.\n\n> Q6: Limited explorations on model scaling.\n\nA6: Thanks for the comment. Despite currently using only a 2B model with 5 experts, we have already achieved state-of-the-art performance on the majority of tasks. We believe Omni-Mol has strong potential, and we will include scaling results for larger size models in the next version.\n\n> Q7: No mechanism for dynamic prompt generation.\n\nA7: Thanks for the comment. You make a good point regarding the deployment of general-purpose molecular models. In practice, we do apply prompt engineering techniques, for example, calling GPT-4 to refine or polish the instructions. However, these strategies fall outside the scope of the technical contributions of this paper, and thus are not discussed.\n\n[1] Liu, S., Nie, W., Wang, C., Lu, J., Qiao, Z., Liu, L., Tang, J., Xiao, C. and Anandkumar, A., 2023. Multi-modal molecule structure–text model for text-based retrieval and editing. Nature Machine Intelligence, 5(12), pp.1447-1457.\n\n[2] Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, and Peer Bork. \"The SIDER database of drugs and side effects\". In: Nucleic acids research 44.D1 (2015), pp. D1075–D1079."}, {"review_id": "HMletOSobg", "reviewer": "Reviewer_m1Tk", "summary": "This work presents Omni-Mol, a model for molecular tasks enabling textual- and molecular inputs and outputs. Omni-Mol uses an LLM-based backbone and further includes a GNN-based graph tokenizer, as well as two methods for handling task-diversity common in molecular learning: GAL is a LoRA variant which can more flexibly learn the optimal rank for different tasks, and MoGE is a MoE approach which allows for different experts to handle different input modalities. Omni-Mol always performs among the best models in a variety of tasks and is ablated to study the effects of different architectural design choices.", "questions": "- Q1: Can the authors please provide information about the runtime needed to train Omni-Mol?\n- Q2: Can the authors elaborate on the necessity for the GNN encoder?\n- Q3: Could the authors explain whether simply treating SELFIES as text and fine-tuning an LLM on interleaved natural language and SELFIES could be a competitive approach to Omni-Mol, or whether such a comparison was considered?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- S1: The work considers a very flexible setting which allows one to interleave natural language, with graph- as well as SELFIES-based molecular information.\n- S2: It is an interesting finding that molecular models benefit from MoE to handle multiple input modalities.\n- S3: The topic is very interesting and the move towards flexible multi-modal molecular models is important and timely.\n\n### Weaknesses\n- W1: It does not become clear whether the GNN-based encoder is truly essential for Omni-Mol. This part of the framework is not properly ablated, yet it seems that training Omni-Mol would become much simpler without the additional GNN representation. The paper justifies this design choices poorly. The authors should at least properly justify this design choice in plain text and ideally perform an ablation study to identify whether the GNN is essential for Omni-Mol. Perhaps I am also overlooking that the GNN is architecturally necessary. In that case, the authors should explain this more clearly in the main paper.\n- W2: The experimental results lack error bars or standard deviation. This is despite the fact that the authors claim in the paper checklist that they consider statistical significance. Without standard deviations, the significance arising from the experimental results is questionable. I strongly suggest that the authors repeat the downstream tasks on multiple random seeds to estimate the standard deviation. While the authors do not specify the training time required for one training run of Omni-Mol, I would understand that training multiple models is not feasible within the time of rebuttal. In that case, I recommend to at least run the inference, which inhibits some stochasticity due to the decoder loop of the LLM, multiple times, as described here: https://arxiv.org/abs/2411.00640.\n- W3: In my opinion, the paper lacks a very important baseline: Fine-tuning the same backbone LLM as used in Omni-Mol, using the same data as Omni-Mol, simply treating SELFIES as text. As far as I understand, this baseline is not considered, yet would allow to estimate the actual improvement arising from treating multiple molecular modalities differently.", "was_revised": true, "final_justification": "The authors have provided a very solid rebuttal overall and largely addressed my concerns. Overall I find this work to be highly relevant,  and the experimental study conducted well, in particular after error bars and the LLM-only baseline were added during the rebuttal. As a result, I have decided to increase my score.", "rebuttal": "> Q1: Ablation on GNN-based encoder.\n\nA1: Thanks for the comment. We follow a long established line of works [1, 2, 3] that uses GNN features as multi-modal feature, which can enhance the model’s ability to understand molecules’ stuctures. To further validate the role of the GNN component within our framework, we conducted an additional ablation study. Due to time constraints, the experiments were carried out on a subset of the tasks. Specifically, we remove the graph feature derived by projector and re-train a pure text model that uses only features from LLM’s word embedding, the results are shown below.\n\nMolcap\n| Method    | BLEU-2 | BLEU-4 | METEOR | ROUGE-1 | ROUGE-2 | ROUGE-L |\n|-----------|--------|--------|--------|---------|---------|---------|\n| Omni-Mol   | 0.511  | 0.421  | 0.556  | 0.593   | 0.434   | 0.530   |\n| w/o GNN   | 0.495  | 0.408  | 0.542  | 0.582   | 0.423   | 0.521   |\n\nReagent\n| Method    | Exact  | BLEU  | RDK   | MACCS | Morgan | Leven   | val  |\n|-----------|--------|-------|-------|-------|--------|---------|------|\n| Omni-Mol   | 0.266  | 0.749 | 0.586 | 0.651 | 0.542  | 14.026  | 1.000 |\n| w/o GNN   | 0.258  | 0.738 | 0.591 | 0.650 | 0.546  | 14.126  | 1.000 |\n\nRetrosynthesis\n| Method    | Exact  | BLEU  | RDK   | MACCS | Morgan | Leven  | val  |\n|-----------|--------|-------|-------|-------|--------|--------|------|\n| Omni-Mol   | 0.594  | 0.962 | 0.861 | 0.910 | 0.828  | 8.386  | 1.00 |\n| w/o GNN   | 0.577  | 0.963 | 0.855 | 0.909 | 0.822  | 8.779  | 1.000 |\n\nYield\n| Method    | BH     | SM     |\n|-----------|--------|--------|\n| Omni-Mol   | 0.953  | 0.688  |\n| w/o GNN   | 0.9025 | 0.6299 |\n\nCatalyst\n| Model     | Exact  | BLEU  | RDK   | MACCS | Morgan | Leven | val  |\n|-----------|--------|-------|-------|-------|--------|-------|------|\n| Omni-Mol   | 0.742  | 0.794 | 0.911 | 0.899 | 0.747  | 1.843 | 1    |\n| w/o GNN   | 0.744  | 0.845 | 0.906 | 0.885 | 0.750  | 1.784 | 1.000 |\n\nQuantum Mechanics Property Prediction\n| Method    | HOMO  | LUMO  | GAP   | AVG   |\n|-----------|-------|-------|-------|-------|\n| Omni-Mol   | 0.0038| 0.0047| 0.0049| 0.0044|\n| w/o GNN   | 0.0038| 0.0037| 0.0044| 0.0040|\n\nWe can see that Omni-Mol benefits significantly from the GNN encoder on Molcap, Reagent, Yield etc., which particularly rely on the structural information of molecules. For other tasks, we observe similar performance with or without the GNN, indicating that Omni-Mol relies more heavily on textual information to generate responses in those scenarios.\n\n> Q2: The experimental results lack error bars or standard deviation.\n\nA2: Thanks for the comment. Following the guidance provided in the cited paper, we varied the sampling temperature and random seed to perform multiple inference runs. The experimental results are shown below.\n\n|Task|Exact|BLEU|LEVEN|MACCS|RDK|MORGAN|Val|\n|-|-|-|-|-|-|-|-|\n|Forward|0.763±0.001|0.982±0.001|5.282±0.055|0.949±0.000|0.906±0.001|0.886±0.001|1±0.000|\n|Reagent|0.276±0.004|0.742±0.002|14.033±0.070|0.660±0.003|0.596±0.004|0.555±0.003|0.999±0.000|\n|Retrosynthesis|0.574±0.002|0.961±0.000|8.325±0.067|0.915±0.001|0.870±0.001|0.835±0.001|0.999±0.000|\n|Solvent|0.548±0.001|0.771±0.001|2.603±0.008|0.692±0.001|0.694±0.002|0.666±0.002|1±0.000|\n|Catalyst|0.746±0.003|0.781±0.003|2.153±0.068|0.893±0.002|0.913±0.001|0.751±0.003|1±0.000|\n|TextGuidedMolGen|0.133±0.001|0.834±0.001|22.324±0.123|0.730±0.001|0.576±0.002|0.452±0.002|0.961±0.001|\n\n|Task|LOGP|TPSA|WEIGHT|\n|-|-|-|-|\n|Mol2Num|0.460±0.002|6.225±0.058|11.257±0.031|\n\n|Task|HOMO|LUMO|GAP|AVG|\n|-|-|-|-|-|\n|Quantum Mechanics|0.0038±0.000|0.0038±0.000|0.0047±0.000|0.0041±0.000|\n\n|Task|BH|SM|\n|-|-|-|\n|Yield|0.898±0.031|0.632±0.013|\n\n|Task|BLEU-2|BLEU-4|METEOR|ROUGE-1|ROUGE-2|ROUGE-L|\n|-|-|-|-|-|-|-|\n|Molcap|0.536±0.001|0.447±0.001|0.574±0.001|0.605±0.001|0.449±0.001|0.542±0.001|\n|Experiment|0.574±0.001|0.450±0.001|-|0.533±0.001|0.275±0.001|0.466±0.001|\n\n> Q3: Lacks a baseline with pure text.\n\nA3: Thanks for the comment. Please refer to A1 above.\n\n> Q4: Runtime to train Omni-Mol\n\nA4: Thanks for the comment. The runtime for 15 tasks in total is 3 * 24 * 8 = 576 A100 80G GPU hours.\n\n[1] Cao, H., Shao, Y., Liu, Z., Liu, Z., Tang, X., Yao, Y. and Li, Y., 2024. PRESTO: Progressive pretraining enhances synthetic chemistry outcomes. arXiv preprint arXiv:2406.13193.\n \n[2] Li, S., Liu, Z., Luo, Y., Wang, X., He, X., Kawaguchi, K., Chua, T.S. and Tian, Q., 2024. Towards 3d molecule-text interpretation in language models. arXiv preprint arXiv:2401.13923.\n \n[3] Cao, H., Liu, Z., Lu, X., Yao, Y. and Li, Y., 2023. Instructmol: Multi-modal integration for building a versatile and reliable molecular assistant in drug discovery. arXiv preprint arXiv:2311.16208."}, {"review_id": "cQrPmaXygn", "reviewer": "Reviewer_8Wa4", "summary": "This paper aims to build a general-purpose, multi-task molecular model based on a large language model. To this end, the authors mainly focus on and propose the following:\n- A multi-task dataset of 16 molecular tasks collected from public databases by classifying them into four folds: Mol2Mol, Mol2Text, Mol2Num, and Text2Mol. \n- Gradient adaptive LoRA (GAL), which dynamically adjusts the scaling factor with learnable parameters to enhance the multi-task learning.\n- Mixture-of-GAL-Experts, a MoE architecture with the proposed GALs.\n\nThe resulting model, Omni-Mol, is evaluated on the collected multi-task dataset, outperforming the baselines.", "questions": "- How important is clipping in GLA? What happens if the clipping is removed?\n- Why are other tasks, such as molecular classification, not considered in this work?\n- Could the authors provide the experimental results on the classification tasks without fine-tuning?\n- What do the left and right y-axes denote in Figure 5 (Middle) and (Right)?", "limitations": "The main limitation of this work is that the proposed model, Omni-Mol, does not show its ability to generalize to tasks beyond the training dataset. I highly recommend evaluating the proposed model on other benchmarks. Additionally, the design of the molecular representations is not reasonable, leaving critical problems such as the permutation equivariance of graph-structured data and diverse forms of string representations.", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 2, "originality": 1, "strengths_and_weaknesses": "**Strengths**\nLeveraging MoE architectures is beneficial for improving performance.\n- In the ablation study, GAL demonstrates performance improvement, indicating its effectiveness in task learning.\n\n**Weaknesses**\n- Omni-Mol is not evaluated on the general tasks beyond the collected dataset. In other words, the proposed model does not exhibit its generalization ability to diverse tasks. Even though the main motivation of this paper is to build a general molecular LLM, the experimental results are limited to showing its effectiveness on its own dataset, not addressing the tackled problem.\n- The proposed method does not guarantee the permutation equivariance of graph-structured data as it directly forwards the node embeddings of graph representations to LLMs. Therefore, if the nodes are permuted, the responses of Omni-Mol will be significantly changed, often generating irrelevant results.\n- As LLMs better understand SMILES than SELFIES, leveraging SELFIES does not fully utilize the internal knowledge in LLMs, which could show inferior performance compared to leveraging SMILES. Even though SELFIES is beneficial to generate valid molecules, using SMILES brings more practicality by fully leveraging the internal knowledge of LLMs and not degrading the original LLMs' abilities.\n- SMILES and SELFIES are not unique, as they could have diverse forms. Specifically, SMILES has diverse forms [1], and as SELFIES is derived from the ordering of SMILES, SELFIES is also not unique. Therefore, using SELFIES could lead misunderstanding of molecular modalities.\n- It is not clear why the multimodal projector is trained only on PubChem. To achieve multi-task learning, training on all the collected datasets sounds more reasonable.\n- The motivation of GLA is not well-established. Motivation experiment for GLA shows that the optimal rank in LoRA could be different according to the tasks. However, it is not directly related to the dynamic scaling factor, as the rank is still static. \n- Related works [2,3,4] are not experimentally compared.\n- The proposed dataset is a collection of well-known instruction datasets, of which the novelty is limited.\n\n\n\n[1] Ganeeva, Veronika, et al. \"Chemical language models have problems with chemistry: A case study on molecule captioning task.\" The Second Tiny Papers Track at ICLR, 2024.\n\n[2] Park, Jinyoung, et al. \"LLaMo: Large Language Model-based Molecular Graph Assistant.\" NeurIPS, 2024.\n\n[3] Zhang, Juzheng, et al. \"UniMoT: Unified molecule-text language model with discrete token representation.\" ArXiv, 2024.\n\n[4] Kim, Dongki, et al. \"Mol-LLaMA: Towards general understanding of molecules in large molecular language model.\" ArXiv, 2025.", "was_revised": true, "final_justification": "I acknowledge that SELFIES is beneficial in certain cases, that the training of the multimodal projector is reasonable, that the proposed model architecture (GAL) is effective in addressing the targeted points, and that the proposed Omni-Mol model is robust to permutation.\n\nHowever, I still have concerns about the following points:\n\nOriginality of the proposed dataset: While the authors have made efforts in curating the dataset, including adding instructions, unification, and de-duplication, it is primarily composed of existing curated datasets, not introducing a truly novel task.\nGeneralizability: Although the proposed model is designed for generalizability, its performance is moderate on OOD tasks.\nOverall, I raise my score accordingly.", "rebuttal": "> Q1: No evaluation on generalization ability to diverse tasks. \n\nA1: Thanks for the comment, the main motivation of Omni-Mol is to train a single model capable of solving multiple tasks, in contrast of using dedicated adapters for different tasks. However, we agree with you that generalizing to OOD data is indeed an interesting aspect. Thus, we directly implement the zero-shot evaluation on three different classification datasets. Please refer to Reviewer fT5E A4.\n\n> Q2: Doubt on permutation equivariance.\n\nA2: Thanks for the comment. First, the graph features obtained from the GNN encoder are aligned with the textual input. The graph-based model we adopt, *MoleculeSTM [1]*, has undergone extensive pretraining for graph-text alignment, enabling it to provide the LLM with sufficient structural information from the molecular graph. Therefore, permuting the node order does not significantly impact performance. However, to provide more rigorous evidence, we shuffle the order of the nodes and evaluated the performance of Omni-Mol. This was done by applying random permutation to the graph data. The experimental results are as follows:\n\nForward\n||Exact|BLEU|LEVEN|MACCS|RDK|MORGAN|Val|\n|-|-|-|-|-|-|-|-|\n|Permut.|0.726|0.9817|5.595|0.9482|0.8942|0.8683|1|\n|noPermut.|0.73|0.98|5.55|0.947|0.895|0.87|1|\n\nReagent\n||Exact|BLEU|LEVEN|MACCS|RDK|MORGAN|Val|\n|-|-|-|-|-|-|-|-|\n|Permut.|0.23|0.7131|14.822|0.6265|0.556|0.519|0.99|\n|noPermut.|0.23|0.726|14.59|0.627|0.557|0.52|1|\n\nRetrosynthesis\n||Exact|BLEU|LEVEN|MACCS|RDK|MORGAN|Val|\n|-|-|-|-|-|-|-|-|\n|Permut.|0.567|0.959|9.075|0.908|0.862|0.8284|1|\n|noPermut.|0.57|0.96|8.97|0.909|0.864|0.83|1|\n\nSolvent\n||Exact|BLEU|LEVEN|MACCS|RDK|MORGAN|Val|\n|-|-|-|-|-|-|-|-|\n|Permut.|0.517|0.756|2.72|0.670|0.671|0.639|1|\n|noPermut.|0.52|0.759|2.71|0.671|0.673|0.64|1|\n\nCatalyst\n||Exact|BLEU|LEVEN|MACCS|RDK|MORGAN|Val|\n|-|-|-|-|-|-|-|-|\n|Permut.|0.714|0.782|1.9731|0.889|0.906|0.714|1|\n|noPermut.|0.72|0.792|1.96|0.886|0.904|0.72|1|\n\nMol2Num\n||LOGP|TPSA|WEIGHT|\n|-|-|-|-|\n|Permut.|0.503|11.09|11.357|\n|noPermut.|0.49|11.07|11.07|\n\nQuantum Mechanics Property Prediction\n||HOMO|LUMO|GAP|AVG|\n|-|-|-|-|-|\n|Permut.|0.0038|0.0057|0.0049|0.0048|\n|noPermut.|0.0038|0.0047|0.0049|0.0044|\n\nYield\n||BH|SM|\n|-|-|-|\n|Permut.|0.8808|0.606|\n|noPermut.|0.94|0.68|\n\nExperiment\n||BLEU-2|BLEU-4|ROUGE-1|ROUGE-2|ROUGE-L|\n|-|-|-|-|-|-|\n|Permut.|0.573|0.449|0.532|0.274|0.465|\n|noPermut.|0.572|0.448|0.532|0.274|0.464|\n\nMolcap\n||BLEU-2|BLEU-4|METEOR|ROUGE-1|ROUGE-2|ROUGE-L|\n|-|-|-|-|-|-|-|\n|Permut.|0.493|0.44|0.545|0.602|0.435|0.538|\n|noPermut.|0.529|0.44|0.571|0.604|0.447|0.541|\n\nTextGuidedMolGen\n||Exact|BLEU|LEVEN|MACCS|RDK|MORGAN|Val|\n|-|-|-|-|-|-|-|-|\n|Permut.|0.119|0.820|23.108|0.719|0.559|0.439|0.96|\n|noPermut.|0.12|0.824|24.179|0.721|0.562|0.442|0.963|\n\nWe can draw the conclusion that Omni-Mol is robust to permutation equivariance.\n\n> Q3: Utilization of the internal knowledge in LLMs.\n\nA3: Thanks for the comment. The motivation of using SELFIES is to allow higher success rate when converting the 1D representation into 2D graph and 3D graph. There are a line of works using SELFIES as their 1D molecule representation, as suggested by PRESTO [2], we use SELFIES for its broader applicability. In fact, many of our baseline results are based on SMILES, meaning they fully leverage the pretrained knowledge embedded within the LLM. In contrast, our method achieves SOTA performance without relying on such pretrained domain knowledge, which further highlights the effectiveness of Omni-Mol.\n\n> Q4: SELFIES could lead misunderstanding.\n\nA4: Thanks for the comment. You’ve made a very insightful point, there is no universally perfect molecular representation. We chose SELFIES primarily due to its higher robustness. In fact, we conduct empirical comparisons and observe that SMILES often leads to parsing failures for a significant number of samples. \n\nNumber of failed cases\n|Method|Forward|Retrosynthesis|Reagent|Solvent|\n|-|-|-|-|-|\n|SELFIES|0|0|0|0|\n|SMILES|10|16|2|6|\n\nThe experimental results are shown below, which further justifies our decision to use SELFIES.\n\n> Q5: Multimodal projector training.\n\nA5: Thanks for the comment. The projector is pre-trained on PubChem for early alignment between LLM and the graph features. In the second stage, projector is still activated and can be updated by multi-task training signals.\n\n> Q6: Doubt on the dynamic scaling factor, as the rank is still static.\n\nA6: Thanks for the comment. The rank in LoRA adapters is fixed by design to enable efficient fine-tuning. Our solution is innovative in that it introduces a learnable scaling factor, calculated as $\\alpha / r$, which effectively links the scaling factor to the adapter’s rank. When the intrinsic rank varies across tasks, GAL addresses this issue by automatically scaling the learning weights under specific lora rank, thereby achieving better multi-task balance.\n\n> Q7: Related works [3, 4, 5] are not experimentally compared.\n\nA7: Thanks for the comment, we have cited all the related works to our paper. Due to time limit, we re-implement Mol-Llama as one additional baseline, we noticed that the 3D GNN used by Mol-Llama supports single molecule input only, we selected tasks that involve only one molecule as input, the results are shown below\n\nDQA\n|Method|BLEU-2|BLEU-4|METEOR|ROUGE-1|ROUGE-2|ROUGE-L|\n|-|-|-|-|-|-|-|\n|Omni-Mol|0.552|0.440|0.58|0.53|0.38|0.49|\n|Mol-Llama|0.507|0.428|0.574|0.522|0.374|0.484|\n\nQuantum Mechanics\n|Method|HOMO|LUMO|GAP|AVG|\n|-|-|-|-|-|\n|Omni-Mol|0.0038|0.0047|0.0049|0.0044|\n|Mol-Llama|0.0049|0.0050|0.0049|0.0049|\n\nMol2Num\n|Method|LogP|TPSA|Weight|\n|-|-|-|-|\n|Omni-Mol|0.49|5.89|11.07|\n|Mol-Llama|0.518|5.914|10.591|\n\nMolcap\n|Method|BLEU-2|BLEU-4|METEOR|ROUGE-1|ROUGE-2|ROUGE-L|\n|-|-|-|-|-|-|-|\n|Omni-Mol|0.529|0.440|0.571|0.604|0.447|0.541|\n|Mol-Llama|0.488|0.396|0.528|0.573|0.408|0.509|\n\nOmni-Mol slightly falls short on several Mol2Num tasks but exhibits significantly better performance on other tasks, which fully demonstrates the superiority of Omni-Mol as a general-purpose molecular LLM.\n\n> Q8: Doubt on novelty of datasets.\n\nA8: Thanks for the comment. Our dataset makes a significant contribution to the field in several key aspects: **First**, we collect the most comprehensive set of molecular tasks, augmenting instruction-missing data with carefully constructed prompts, and further enhancing all tasks with diverse instructions. **Second**, we conduct thorough data leakage checks across the entire dataset and remove duplicate samples to build the largest general-purpose molecular dataset to date. **Third**, our dataset covers the broadest and most complete range of task types, laying a solid foundation for the development of general-purpose molecular AI models.\n\n> Q9: Ablation on clipping in GLA.\n\nA9: Thanks for the comment. Clipping is very important in our training. We have added the ablation study results on clipping below. Due to the time limit, we conduct the experiment on a subset of tasks.\n\nCatalyst\n|Model|Exact|BLEU|RDK|MACCS|Morgan|Leven|val|\n|-|-|-|-|-|-|-|-|\n|Omni-Mol|0.742|0.794|0.911|0.899|0.747|1.843|1|\n|w/o clip|0.731|0.775|0.898|0.878|0.737|2.144|1|\n\nForward\n|Model|Exact|BLEU|RDK|MACCS|Morgan|Leven|val|\n|-|-|-|-|-|-|-|-|\n|Omni-Mol|0.738|0.985|0.884|0.941|0.865|6.06|1|\n|w/o clip|0.703|0.980|0.865|0.927|0.840|7.045|1|\n\nQuantum Mechanics\n|Method|HOMO|LUMO|GAP|AVG|\n|-|-|-|-|-|\n|Omni-Mol|0.0038|0.0047|0.0049|0.0044|\n|w/o clip|0.0039|0.0055|0.0047|0.0047|\n\nMolcap\n|Method|BLEU-2|BLEU-4|METEOR|ROUGE-1|ROUGE-2|ROUGE-L|\n|-|-|-|-|-|-|-|\n|Omni-Mol|0.511|0.421|0.556|0.593|0.434|0.530|\n|w/o clip|0.483|0.391|0.526|0.570|0.406|0.507|\n\nReagent\n|Method|Exact|BLEU|RDK|MACCS|Morgan|Leven|val|\n|-|-|-|-|-|-|-|-|\n|Omni-Mol|0.266|0.749|0.586|0.651|0.542|14.026|1|\n|w/o clip|0.256|0.736|0.577|0.645|0.533|14.619|1|\n\nRetrosynthesis\n|Method|Exact|BLEU|RDK|MACCS|Morgan|Leven|val|\n|-|-|-|-|-|-|-|-|\n|Omni-Mol|0.594|0.962|0.861|0.910|0.828|8.386|1|\n|w/o clip|0.542|0.959|0.838|0.897|0.802|9.937|1|\n\nSolvent\n|Method|Exact|BLEU|RDK|MACCS|Morgan|Leven|val|\n|-|-|-|-|-|-|-|-|\n|Omni-Mol|0.554|0.782|0.703|0.700|0.678|2.498|1|\n|w/o clip|0.550|0.775|0.692|0.693|0.667|2.550|1|\n\nYield\n|Method|BH|SM|\n|-|-|-|\n|Omni-Mol|0.953|0.688|\n|w/o clip|0.933|0.680|\n\nFrom the results, we observe performance drops across several tasks, which aligns with our assumption that different tasks have different optimal rankings. \n\n> Q10: Why are other tasks, such as molecular classification, not considered in this work?\n\nA10: Thanks for the comment. First, the DQA task includes approximately 10k entries related to molecular graph classification. For example, there is a sample with instruction: “What is dysidenin classified as chemically?”, and output: “Dysidenin is classified as an organochlorine compound.” Therefore, strictly speaking, we categorize it under the text2text setting. \n\n> Q12: What do the left and right y-axes denote in Figure 5 (Middle) and (Right)?\n\nA12: Thanks for the comment. The left y-axis displays average performance excluding yield prediction, while the right y-axis shows yield prediction performance. These were separated due to substantial scale differences that would impair visualization.\n\n[1] Liu, S., Nie, W., Wang, C., Lu, J., Qiao, Z., Liu, L., Tang, J., Xiao, C. and Anandkumar, A., 2023. Multi-modal molecule structure–text model for text-based retrieval and editing. Nature Machine Intelligence, 5(12), pp.1447-1457. \n\n[2] Cao, H., Shao, Y., Liu, Z., Liu, Z., Tang, X., Yao, Y. and Li, Y., 2024. PRESTO: Progressive pretraining enhances synthetic chemistry outcomes. arXiv preprint arXiv:2406.13193.\n\n[3] Park, Jinyoung, et al. \"LLaMo: Large Language Model-based Molecular Graph Assistant.\" NeurIPS, 2024.\n\n[4] Zhang, Juzheng, et al. \"UniMoT: Unified molecule-text language model with discrete token representation.\" ArXiv, 2024.\n\n[5] Kim, Dongki, et al. \"Mol-LLaMA: Towards general understanding of molecules in large molecular language model.\" ArXiv, 2025."}, {"review_id": "avARhV61Yr", "reviewer": "Reviewer_FtzT", "summary": "The authors propose  OmniMol, a generalist chemistry LLM with the ability to solve a wide range of tasks. OmniMol is a Llama 3.2 model finetuned with a novel LORA-based approach designed for multitask learning that solves Text2Mol, Mol2Text, Mol2Mol and Mol2Num tasks. Experiments with up to 8B models were done on a multitask dataset with a total of 1.4M samples", "questions": "- I have some concerns on the results. (1) In some tables OmniMol is incorrectly bolded  (e.g., Molecular Captioning Task, it is not the best). (2) It seems that some cherry-picking on the competing methods was done, e.g., the Text+Chem T5 model achieved better performance on the two tasks (Mol2Text and Text2Mol) of Chebi-20.\n\t- I had a look at the code but it has no README which hampers reproducibility. Also, will OmniMol be available publicly, e.g., via a HF spaces?\n\t- The section on SMILES vs SELFIES gives the wrong perception SMILES may often cause errors in RDKit. I think you want to say that such errors occur frequently *when generating SMILES with a LM*. Please be more explicit here. Also, I think that a small ablation on SMILES vs SELFIES would be useful because only ~55% of the samples have SMILES/SELFIES output whereas ~90% have SMILES/SELFIES input. Then, when you tokenize SELFIES with the BPE-based Lama tokenizer, you may distort the structure of the SELFIES such that not all generated SELFIES are valid anymore. E.g., when you tokenize [C][C] into \"[C\" \"][ \"C\" and \"]\" you arrive at a vocab that can still be used to produce invalid SELFIES like \"][]\". \n\t- P4 L150: Do you mean the hidden size of the LLM's nn.Embedding and the GNN's node embedding size? If really only the sequence lengths differ you just stack horizontally and no padding is necessary (apart from standard padding when you form batches). \n\t- Figure 4: I am buying the scalability argument but the visualization is unfortunate as the bar seems flat -- I would visualize this on a relative scale (y-axis is the delta compared to the first value)", "limitations": "There is a very generic and shallow section about limitations in the appendix.\nThe main paper lacks a substantial discussion of its own limitations", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The significance of this research topic is high since generalist chemistry LLMs would benefit the CS as well as the chemistry community well. \n\nThe originality seems medium given that many others have tackled this problem before in a similar fashion. The methodological delta to existing methods is small. E.g., even the early Text+ChemT5 work named as \"pioneering\" by the authors can already do all four mentioned tasks natively (Text+Chem T5 is not explicitly evaluated on Mol2Num but it can solve it naturally without any architectural modification).\n\nOn the positive side, the authors propose a custom LORA-based method (MoGE) which boosts performance. This approach and its usage in a MOE settings is well-motivated and interesting, especially also the ablation study on its value compared to vanilla LORA. \n\nClarity is good overall but can be improved, e.g., the figure ordering is random and they are not referenced in order. \n\nThe results are generally convincingly presented but I have concerns that some of them are cherry-picked (see below)", "was_revised": true, "final_justification": "While I still believe that the methodological novelty is relatively low and OmniMol does not really unlock new capabilities of LLMs in chemistry, the rebuttal helped me to understand better the richness of the various datasets and the effort in homogenizing and providing this data to the community. Moreover, the scale of the model is significant (considering standard academic resources) and performance gains to previous methods seem robust. Therefore, I increased my score from 3 to 4.", "rebuttal": ">Q1: The methodological delta to existing methods is small. E.g., even the early Text+ChemT5 work named as \"pioneering\" by the authors can already do all four mentioned tasks natively (Text+Chem T5 is not explicitly evaluated on Mol2Num but it can solve it naturally without any architectural modification).\n\nA1: Thanks for the comment. In fact, on our comprehensive general-purpose molecular task dataset, no existing baseline has been able to fully handle the range of tasks. The main challenge lies in the instability of LLM training when faced with highly diverse task types—simply adding more task-specific data does not lead to effective scaling. OmniMol is the first to address this problem by introducing GAL-MoE, a carefully designed mechanism that enables stable and scalable multi-task learning. We also appreciate your mention of Text-Chem T5, and we will provide a detailed comparison with it in the following response.\n\n>Q2: Clarity is good overall but can be improved, e.g., the figure ordering is random and they are not referenced in order.\n\nA2: Thanks for the comment. We will adjust the layout of our paper.\n\n>Q3: In some tables OmniMol is incorrectly bolded.\n\nA3: Thanks for the comment. We have noticed the typo and revised it.\n\n>Q4: Cherry-picking on the competing methods.\n\nA4: Thanks for the comment. We appreciate the reviewer for bringing up this paper, but we respectively disagree with the implication that we are cherry-picking. We had already taken note of this work and reproduced it on our dataset; however, the reproduced results turned out to be quite poor. The detailed experimental results are shown below.\n\nForward Pred\n\n| Model | Exact | BLEU | LEVEN | MACCS | RDK | MORGAN | Val |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.3036 | 0.8808 | 17.2164 | 0.8066 | 0.6456 | 0.5904 | 0.99 |\n| Omni-Mol | 0.73 | 0.980 | 5.55 | 0.947 | 0.895 | 0.87 | 1.00 |\n\nReagent Pred\n\n| Model | Exact | BLEU | LEVEN | MACCS | RDK | MORGAN | Val |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.0437 | 0.5206 | 25.711 | 0.4735 | 0.3417 | 0.3061 | 0.98 |\n| Omni-Mol | 0.23 | 0.726 | 14.59 | 0.557 | 0.627 | 0.52 | 1.00 |\n\nRetrosynthesis\n\n| Model | Exact | BLEU | LEVEN | MACCS | RDK | MORGAN | Val |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.1538 | 0.7965 | 23.601 | 0.7813 | 0.6465 | 0.5661 | 0.99 |\n| Omni-Mol | 0.57 | 0.960 | 8.97 | 0.909 | 0.864 | 0.83 | 1.00 |\n\nSolvent Pred\n\n| Model | Exact | BLEU | LEVEN | MACCS | RDK | MORGAN | Val |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.2694 | 0.6114 | 4.144 | 0.4467 | 0.4196 | 0.3785 | 1 |\n| Omni-Mol | 0.52 | 0.759 | 2.71 | 0.673 | 0.671 | 0.64 | 1 |\n\nCatalyst Pred\n\n| Model | Exact | BLEU | LEVEN | MACCS | RDK | MORGAN | Val |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.5959 | 0.5988 | 3.9208 | 0.8110 | 0.8221 |  0.6069 | 0.99 |\n| Omni-Mol | 0.72 | 0.792 | 1.96 | 0.886 | 0.904 | 0.72 | 1.00 |\n\nQuantum Mechanics Property Prediction Task\n\n| Model | HOMO | LUMO | GAP | AVG |\n| --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.0083 | 0.0095 | 0.0108 |  0.0095 |\n| Omni-Mol | 0.0038 | 0.0047 | 0.0049 | 0.0044 |\n\nMol2Num\n\n| Model | logP | TPSA | Weight |\n| --- | --- | --- | --- |\n| Text-Chem T5 | 2.346 | 39.771 | 85.472 |\n| Omni-Mol | 0.49 | 5.89 | 11.07 |\n\nIUPAC2SELFIES\n\n| Model | Exact | BLEU | LEVEN | MACCS | RDK | MORGAN | Val |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.0204 | 0.7097 | 28.9402 | 0.6792 | 0.4149 |  0.3302 | 0.96 |\n| Omni-Mol | 0.39 | 0.952 | 13.38 | 0.871 | 0.729 | 0.69 | 0.996 |\n\nDescription Q&A Task\n\n| Model | BLEU-2 | BLEU-4 | METEOR | ROUGE-1 | ROUGE-2 | ROUGE-L |\n| --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.4971 | 0.4201 | 0.5434 | 0.5272 | 0.3770 | 0.4938 |\n| Omni-Mol | 0.52 | 0.44 | 0.58 | 0.53 | 0.38 | 0.49 |\n\nExperimental Procedure Prediction\n\n| Model | BLEU-2 | BLEU-4 | ROUGE-1 | ROUGE-2 | ROUGE-L |\n| --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.5733 | 0.4704 | 0.5405 | 0.2831 | 0.4794 |\n| Omni-Mol | 0.572 | 0.448 | 0.532 | 0.274 | 0.464 |\n\nMolecule Captioning\n\n| Model | BLEU-2 | BLEU-4 | METEOR | ROUGE-1 | ROUGE-2 | ROUGE-L |\n| --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.3324 | 0.2408 | 0.3871 | 0.4543 | 0.2765 | 0.3991 |\n| Omni-Mol | 0.529 | 0.440 | 0.571 | 0.604 | 0.447 | 0.541 |\n\nText Guided Molecule Generation\n\n| Model | Exact | BLEU | LEVEN | MACCS | RDK | MORGAN | Val |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Text-Chem T5 | 0.0014 | 0.6241 | 43.574 | 0.5082 | 0.2961 | 0.1830 | 0.72 |\n| Omni-Mol | 0.12 | 0.824 | 23.59 | 0.721 | 0.562 | 0.442 | 0.963 |\n\nYield Prediction\n\n| Model | BH | SM |\n| --- | --- | --- |\n| Text-Chem T5 | 0.171 | 0.151 |\n| Omni-Mol | 0.94 | 0.68 |\n\nTo ensure fairness and transparency, we have publicly released all model files, hyperparameter configurations, training procedures, and training logs for Text-Chem T5 in our GitHub repository (TextChemT5-v0-main folder). In the above table, Following the Text-Chem T5 paper, we fine-tuned its parameters to ensure the best possible performance. Evidently, Omni-Mol outperforms Text-Chem T5 on almost all tasks by a significant margin. If you have any insights or results from reproducing this work, we would be happy to discuss and compare findings in the subsequent stages.\n\n> Q5: Limited reproducibility.\n\nA5: Thanks for the comment and interest. After submitting the paper, we conduct a thorough audit of our code to ensure its correctness and reproducibility. We are highly committed to open-source practices. If you revisit our repository now, you will find that we have updated the code with refined implementation details, added comprehensive comments, provided a complete and well-documented training pipeline, and released our model weights. This allows for a smooth and fully reproducible replication of our results.\n\n> Q6: Wrong perception SMILES may often cause errors in RDKit.\n\nA6: Thanks for the comment. The clear statement is that when an LLM outputs SMILES, it is more prone to generating errors, which can result in outputs that cannot be parsed into valid molecules.\n\n> Q7: Then, when you tokenize SELFIES with the BPE-based Lama tokenizer, you may distort the structure of the SELFIES such that not all generated SELFIES are valid anymore. E.g., when you tokenize [C][C] into \"[C\" \"][ \"C\" and \"]\" you arrive at a vocab that can still be used to produce invalid SELFIES like \"][]\".\n\nA7: Thanks for pointing out, indeed, there is no perfect 1D representation for molecules, we use SELFIES for its robustness and broader applicability. In fact, many of our baseline results are based on SMILES, meaning they fully leverage the pretrained knowledge embedded within the LLM. In contrast, our method achieves SOTA performance without relying on such pretrained domain knowledge, which further highlights the effectiveness of Omni-Mol.\n\n> Q8: P4 L150: Do you mean the hidden size of the LLM's nn.Embedding and the GNN's node embedding size? If really only the sequence lengths differ you just stack horizontally and no padding is necessary (apart from standard padding when you form batches).\n\nA8: Thanks for the question, actually, the padding is due to the varying seuqence length, i.e. differnet number of nodes in molecular embeddings from GNN. We pad them into a regular shape to allow parallel training.\n\n> Q9: Figure 4: I am buying the scalability argument but the visualization is unfortunate as the bar seems flat -- I would visualize this on a relative scale (y-axis is the delta compared to the first value)\n\nA9: Thanks for the advice, we will adjust the figure to enable better visualization\n\n> Q10: Add useful ablation on SMILES vs SELFIES. \n\nA10: Please refer to  Reviewer 8Wa4 A4"}]

Author rebuttal:

Meta-review: This paper introduces a multimodal LLM framework supporting four categories of molecular tasks (Mol2Mol, Mol2Text, Mol2Num, Text2Mol), enabled by an “any-to-any” modality design. The work also proposes Gradient Adaptive LoRA (GAL) to enhance adaptability in multi-task learning and contributes a large-scale dataset of over 1.3M samples across 16 tasks. The topic is timely and significant, as generalist molecular LLMs have the potential to benefit both computer science and chemistry communities.

Strengths include the flexible multimodal setting, the use of MoE architectures, and a well-motivated GAL approach with clear ablation studies. The paper is generally well-written and presents results convincingly. Weaknesses include modest originality, as prior work (e.g., Text+ChemT5) can already perform most of the proposed tasks, some concerns about potential cherry-picked results, and minor clarity issues with figure organization.

Overall, this is an interesting and valuable contribution that advances the development of multimodal molecular LLMs. Most reviewers recommend acceptance.
Decision: Accept (poster)