Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model
Authors: Dongki Kim
Wonbin Lee
Sung Ju Hwang
Venue: NeurIPS 2025 poster ()
Keywords: Large Molecular Language Model
General-purpose Assistant for Molecular Analysis
Molecular Comprehension
Primary area: machine_learning_for_sciences
TL;DR: We propose a large molecular language model with general knowledge and reasoning capabilities for molecular reasoning, demonstrating its utility as a general-purpose assistant for molecular analysis.

Abstract: Understanding molecules is key to understanding organisms and driving advances in drug discovery, requiring interdisciplinary knowledge across chemistry and biology. Although large molecular language models have achieved notable success in task transfer, they often struggle to accurately analyze molecular features due to limited knowledge and reasoning capabilities. To address this issue, we present Mol-LLaMA, a large molecular language model that grasps the general knowledge centered on molecules and exhibits explainability and reasoning ability. To this end, we design key data types that encompass the fundamental molecular features, taking into account the essential abilities for molecular reasoning. Further, to improve molecular understanding, we propose a module that integrates complementary information from different molecular encoders, leveraging the distinct advantages of molecular representations. Our experimental results demonstrate that Mol-LLaMA is capable of comprehending the general features of molecules and providing informative responses, implying its potential as a general-purpose assistant for molecular analysis. Our project page is at https://mol-llama.github.io/.

Mol-LLaMA:
Towards General Understanding of Molecules in Large Molecular Language Model
Dongki Kim', Wonbin Lee', Sung Hwang 1.2
KAIST', DeepAuto.ai
fcleverki, smilelwb01, sungju.hvang) ac.
Abstract
Understanding molecules is key to understanding organisms and driving advances in drug discovery, requiring interdisciplinary knowledge across chemistry and biology. Although large molecular language models have achieved notable success in task transfer, they often struggle to accurately analyze molecular features due to limited knowledge and reasoning capabilities. To address this issue, we present Mol-LLaMA, a large molecular language model that grasps the general knowledge centered on molecules and exhibits explainability and reasoning ability. To this end, we design key data types that encompass the fundamental molecular features, tak- ing into account the essential abilities for molecular reasoning. Further, to improve molecular understanding, we propose a module that integrates complementary in- formation from different molecular encoders, leveraging the distinet advantages of molecular representations. Our experimental results demonstrate that Mol-LLaMA is capable of comprehending the general features of molecules and providing informative responses, implying its potential as a general-purpost assistant for molecular analysis. Our project page is at https : /mol- 10s--laam 1lama, gi github.io/
Introduction
Understanding molecules and their properties is fundamental understanding chemical compounds and living organisms, driving scientific discovery. However, it remains challenging due to the com- plexity of molecules and their behaviors, necessitating comprehensive understanding of molecules. Recent advancements in large language models (LLMs) [46, 16, 39, 40] have demonstrated their potential in understanding core concepts in chemistry and biology [1, 43). Further, LLMS have shown that language can play various roles, which is especially crucial to dealing with substantial complexity in chemistry, biology, and pharmacology by readily obtaining external knowledge from users' prompts. Nevertheless, LLMs struggle to interpret the molecular structures from the raw string representations such as SMILES [52] due to their nature of learning the subword representations.
To integrate structural information into LLMs, recent works have studied molecular LLMs that are instruction-turn tundd with the molecular modality, adopting the multi-modal instruction tuning widely studied in other modalities, such as image [24, 26], audio [13], and video [5, 60, 36]. In the multi- modal instruction tuning, there are molecular structures and 2) the instruction dataset to learn the molecule-related knowledge.
components: 1) the molecular encoder embed the
Despite their success, molecular LLMs still encounter difficulties in understanding and reasoning about the fundamental characteristics of molecules. Based on the observations from Table 1, we identify two key challenges to develop cannera--puross. assistant for molecular analysis as follows:
Molecular LLMs should learn the general features of molecules encompassing their chemical and biological features with a primary emphasis on the basic understanding of structures. Due to their limited knowledge, existing molecular LLMs often misinterpret molecular structures, incorrectly predicting key molecular features, or fail to provide meaningful explanations of molecular features.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Diazepine
Renzodiazzpiin
Quinazoline
Quinoxaline
Input Molecule
Original Description: Bromazepam binds to the GABA receptor GABA-A, causing conformational change and increasing inhibitory effects GABA. Other neurotransmitters not influenced. One of the benzoduuzeiiee used the treatment of anxiety disorders.
User: Explain the features of the given molecule that are most distinct from those of other molecules.
Mol-LLaMA (Ours)
The given molecule a member the benzodiazepine class, which characterized by a fusion of benzene ring and seven-membered diazepine ring. This structure is crucial for its biological activity, particularly its role as sedative and hypnotic agent. ...<omittedd...
characterized by ketone group within the diazepine ring, which is common feature in benzodiazepines. This functional group is important for the interaction with GABA-A receptor, enhancing the inhibitory effects of GABA. <omiitedd
The given molecule
GPT-40
** **Quunazolinee C The molecule contains quinazoline core, .omiteedd.....
Overall, the combination of quinazoline structure with unique substitution patterns, including bromine halogenation and the presence of pyridinyl group, <omittedd.....
LLaMo
The molecule has a benzodiazepine structure with bromo substituent at the 7-position and methyl group the I-position. It is distinct from other molecules in that it contains bromine atom, nitrogen atom, and methyl group, which
present in the other molecules.
3D-MoLM
<omittedd... It belongs to the class of compounds called quinazolines, which are characterized by bicyclic structure consisting of benzene ring fused to pyrimidine ring. .ommiiteed>...
The molecule's structure suggests potential applications in medicinal chemistry, as quinazolines have been found to possess various biological activities, including antitumor, antimicrobial, and anti-inflammatory properties.
Moi-Instructions
The molecule is quinoxaline derivative.
Table 1: Case study to compare molecular understanding and reasoning ability. Mol-LLaMA accurately understands the molecular features, answering a correct molecular taxonomy and providing its distinct properties that are relevant to the given molecule with dctailed explanations. Entire responses are provided in Table 11.
Explainability and reasoning capabilities are essential for molecular reasoning. Unlike other modalities where features directly observable, such image video, molecular features highly complex and are not readily apparent from structures. This complexity necessitates in-depth reasoning based on extensive domain knowledge and interpretability serve general-purpose assistant. However, existing molecular LLMs often fail to provide rationales or perform clear reasoning process, which undermines their ability to accurately predict molecular properties and makes it difficult to assess the reliability of their responses.
The shortcomings of recent molecular LLMs come from their limitations within each component of multi muuti--odal instruction tuning. First, while the instruction tuning with relevant instruction datasets has been shown expand knowledge and introduce reasoning capabilities [58], recent works [30, 3, 12, 61, 57] have primarily relied on public databases which contain knowledge and context restricted specific tasks. Thus, the resulting molecular LLMs often generate irrelevant responses, making wrong predictions of molecular properties and exhibiting limited interpretability. Although the design of instruction datasets that entail domain-relevant knowledge and reasoning capabilities is crucial for molecular reasoning, it still remains underexplored. Second, existing molecular L.LMs [30, 3, 41] rely on single type molecular encoder, limiting them processing only one form of molecular structures, such 2D or 3D molecular graphs. This shortcoming hinders

---

the accurate understanding and prediction of complex molecular features by misinterpretations of molecular structures.
To overcome the aforementioned limitations, we present Mol-LLaMA, a molecular LLM that learns fundamental knowledge centered on molecules with explainability and reasoning ability, positioning it as general-purposs molecular assistant. To this end, we mainly explore the core components lying in multi-modal instruction tuning. First, we construct a novel instruction dataset by designing three key data types that encompass the fundamental features of molecules as well as cultivate the explainability and reasoning ability. Further, to improve the structural understanding, we propose a blending module that integrates complementary information from different molecular representations, alleviating the hallucination problem and enhancing the understanding of molecular features.
We experimentally validate the effectiveness of Mol-LLaMA in understanding the fundamental features of molecules for solving scientific problems, where Mol-LLaMA outperforms baselines including L.MMs and molecular L.MMs, providing accurate, detailed, and helpful responses. We further evaluate Mol-LLaMA on the molecular property prediction task and the molecular comprehension benchmarks, where it not only accurately predicts molecular properties but also generates relevant and helpful responses, highlighting its utility as a general-purposs assistant for molecular analysis. Our contributions can be summarized as follows:
We present Mol-LLaMA that learns general knowledge for molecules across structural, chemical, and biological aspects and exhibits reasoning abilities and explainability, qualified as general- purpose assistant for molecular analysis.
We design three novel data types for the general understanding of molecules, establishing large and informative instruction dataset centered molecular features that can cultivate explainability and reasoning abilities of molecular LLMs.
We devise blending module to fully leverage the complementary information from different types of encoders, alleviating the hallucination and thus enhancing the understanding of molecular structures and advanced features.
The proposed Mol-LLaMA outperforms previous LLMs and molecular LLMs including GPT-40 in the molecular understanding by learning the comprehensive knowledge centered on molecules.
2 Related Works
Molecular Foundation Models Molecular foundation models have achieved remarkable success in modeling molecules using string representations [6, 11, 50, 20). 2D molecular graphs [56, 37], 3D molecular graphs [63, 32, 15, 53] or texts from biomedical literature [17, 23, 2]. Recently, with the emergence of molecule-text pair datasets, multi-modal foundation models have been developed based on contrastive learning [45, 29] or text decoders [10, 59, 34, 7, 28, 42]. However, these models remain limited to perform diverse tasks, which restricts to serve as -purroe molecular assistants.
Large Language Models on Scientific Discovery Large language models (LLMs) [39, 40, 46, 16. 4, 48, 8] have shown that they can play various roles via textual interactions with users. Recently, A14Science and Quantum [1] have demonstrated the promising potential of LLMs in understanding wide range of knowledge and solving complicated scientific problems. Despite their notable progress, LLMs struggle to interpret raw string representations such as SMILES [52] and SELFIES [22], as they learn subword representations that are not well-suited for tokenizing such raw string representations.
Large Molecular Language Models Along with the remarkable progress in multi-modal large language models (LLLMs) [27, 26, 55), molecular LLMMs including MolCA [30], Mol-Instructions [12], LlasMol [57], InstructMol [3], and 3D-MoLM [25] have been developed by training LLMs on molecule-text pair datasets with graph representations modeled by 2D or 3D molecular encoder. In the concurrent work ours, Park et al. [41] propose LLaMo with a projector to seamlessly encode the molecular structures from 2D representations. while training on the public databases and the constructed conversations. On the other hand, Luo al. [35] propose BioMedGPP-LM to integrate the molecular and protein modalities into one LLM by training it on the public database. Despite their promising performance on task transfer, they are trained on the instruction datasets that are typically task-specific, hindering them from functioning as general-purpose assistants. In this work, we aim to build molecular L.IM capable of understanding of general features of molecules with explainability and reasoning capabilities to be utilized general-purpos assistant for molecular analysis.
3

---

3 Mol-LLaMA
We present Mol-LLaMA, a general-purpose assistant for molecular analysis that can comprehend the fundamental features of molecules with explainability and reasoning capabilities.
3.1 Instruction Dataset Construction for Comprehending Molecular Features
The instruction dataset lies at the core of the multi-modal instruction tuning, promoting understanding of other modalities and introducing the reasoning capabilities [27, 26, 58]. In the context of our work, scientific problems are interdisciplinary, necessitating wide-ranging expertise including chemistry and biology, along with complex reasoning skills. To this end, an ideal instruction dataset for molecular LLMs should encompass comprehensive information with detailed explanations of the underlying scientific principles.
Goal
However, existing instruction datasets lack comprehensive information. For example, in the case of the existing data example 1 of Table 2, the context of the target answer is highly limited, failing to provide relevant knowledge and an in-depth explanation of the causality of the target features. Moreover, the knowledge addressed in this data is restricted specific tasks, hindering the learning of fundamental molecular features. Consequently, due to their narrow knowledge scope and limited reasoning capabilities, such instruction data obstructs molecular LLMs from functioning as general- purpose assistants. To overcome these limitations, our goal is to establish an instruction dataset that explicitly presents the fundamental features of molecules and explains their causality.
Data Generation Pipeline for Molecular Reasoning To this end, employ GPT-4o [40] whose capabilities for scientific knowledge and reasoning have been widely studied [1]. We prompt GPT-4o to generate instruction data by leveraging two types of contexts: 1) processed string representations of molecules and 2) their descriptions. For the processed string representations, we use the annotated IUPAC names, which explicitly specify the names of functional groups and their connectivity [14]. For the descriptions, we use annotated descriptions from PubChem [21] to provide grounded features.
The next question is how to design an instruction dataset that covers the fundamental features of molecules while cultivating the reasoning capabilities. To this end, we are inspired by the natural law, where the molecular features have hierarchical relationship. That is, the structures determine the chemical features, while the biological features are determined by both the structural and chemical features. Based our observation, devise three novel data types that address these fundamental features as well as enhance the reasoning capabilities, as shown in Table
Detailed Structural Descríption: We first design detailed descriptions of molecular structures to upskill the foundational understanding of molecules and improve the comprehension of advanced chemical and biological features. To achieve this, we prompt GPT-4o to include explanations of functional groups and their connectivity in detail. For structural descriptions, we use the annotated IUPAC name exclusively, focusing on accurately describing the structural information without emphasizing the molecular functionalities.
2. Atructure-t-Feaaturr Relationship Explanation: To enable comprehensive understanding of chemical and biological features, we create structre-to-featten relationship explanations by directly relating the structural information to its molecular features. Note that learning these relationships inherently helps an LLM understand the causality between molecular structures and their features, allowing it to provide rationales when responding.
3. Comprehensive Conversation: Even though the two types above facilitate the learning of general knowledge about molecules, important to tailor LLMs handle diverse contexts of requests. Thereby, we design conversations to cultivate the ability to handle various inquiries. Specifically, considering the hierarchical relationships of molecular features. we create comprehensive con- versations by instructing to gradually deepen the level of features, progressing from structural features to chemical and biological features, cultivate a comprehensive understanding and the step-by-sier reasoning ability.
To ensure the quality of corruaction-ollowinng samples, we further filter out factually incorrect ones. Inspired by -LMaaa--a [62], we use GPT-4o to evaluate the factual accuracy of the samples and select those with correct content, establishing 284k caatuuttion-ollowiin samples from the

---

Property Prediction Q: Is blood-brain barrier permeability (BBBP) property of this molecule?
Existing Data Example 1:
A: Yes
Methyl
Existing Data Example 2: Description
Cysteine
S-methyleysteine is eysteinc derivative that is L-cysteine in which the hydrogen attached to the sulfur is replaced by methyl group. has role human urinary metabolite and a plant metabolite. is tautomer a S-methylcysteine zwitterion.
Data Type 1: Detailed Structural Descriptions
The molecule provided is chiral amino acid that contains several identifiable substructures.
1. Amino Group**: This functional group consists of nitrogen atom honded to two hydrogen atoms (-NH2). It is attached to the second carbon atom ...oomitedd>...
2. ** Carboxylic Acid Group"": This group includes carbon atom double-bonded to an oxygen atom and also bonded to (-OH) group. It is attached the end of the carbon chain, characteristic of amino acids <o.mitdddd
Data Type 2: Structure-tor Ceettturr-----Ftturr Relationship Explanations
Firstly, the presence of the amino group (-NH2) classifies this compound as an amino acid, which contributes to its ability to act as a zwitterion. This means that it can contain both a positive charge and negative charge <omiited>...
The sulfur atom with a methyl attachment forms a thioether. This increases the molecule's hydropho- bic characteristics compared to cysteine, which contains more polar thiol group. <omitted)>...
Data Type 3: Comprehensive Conversations
USER: Can you tell about the basic structure of the given molecule?
ASSISTANT: The given molecule is derivative of cysteine, where the hydrogen atom attached to the sulfur group is replaced by a methyl group. ...motdtedd...
USER: How does the molecule's structure influence its chemical behavior?
ASSISTANT: The structure of the given molecule, particularly the presenee of both amino and carboxylic acid groups, contributes to its zwitterionic nature physiological pH. <omittedd..
USER: What role does this molecule play biologically?
ASSISTANT: Biologically, S-methyleysteine serves as metabolite in both humans and plants
...comited>...
Table One example the constructed instruction data, two blocks show the data examples public datasets. The following three blocks show data types and corresponding responses of GPT-4o for each data type. Entire responses of the given example are provided in Table 30, 31, and 32 of Appendix B.
training set the PubChem 324k dataset [25]. We refer to the constructed instruction dataset as Mol-LLaaMA-Insteen We note that our dataset not only aids in understanding the molecular features but also enhances explainability and reasoning capabilities by extensively addressing fundamental molecular features and various types of interactions between users and assistant. For more details on the dataset construction, please refer to Appendix B.
3.2 Model Architecture
We now introduce the model architecture of Mol-LLaMA. Here, our goal is to accurately capture structural information, to further improve the understanding of molecular features. To this end, the model architecture consists of four components: molecular encoders, a 2D-3D blending module, projector, and large language models.
Molecular Encoders We observe that each molecular encoder has distinct advantages. While the 2D encoder explicitly models the bond information and their connectivity, the 3D encoder, which represents the molecules as point clouds, captures the spatial arrangements of atoms which is crucial to understanding 3D features such as surface area and volumes. To fully leverage the strength of each encoder, we propose to use both the encoder and 3D encoder. We opt Molecule [29] as the 2D encoder and UniMol [63] as the 3D encoder.
5

---

Module
Output cyctepant meritoocc cotbon
Self-Att
Cross Alt
MX
FFN
Molecular
FFN
FPN
Encoder
Cross-Alt
LoRA
Large Language Model
3D Molecular Encoder 2
Self-Ait
FFN
Cross-Att
FPN
Self-Alt
Language Instruction the smocpioo vito auteortooreett and
Learnable
Query Tokens
Figure/ I: Illustration of the end-to-end instruction luning stage of Mol-LLaMA, It trained on the proposed instruction datascts, where the blending module, Q-Former, and LoRA in LLMs are trained, while the molecular encoders LLM are frozen.
2D-3D Blending Module Since each representation is independently modeled, we propose a blending module that combines these molecular representations using a cross-attention scheme. Specifically, given the molecular embeddings from each encoder which are the concatenation of graph and node embeddings, self-attention and cross-attention sequentially applied blend the complementary information from each encoder. Then, we concatenate the 2D and 3D embeddings before forwarding to the projector, as shown Fig. (Blending Module).
Projector We employ Q-Former [24, 25] to project the unified molecular representations from the proposed 2D-3D blending module LLM, which embeds molecules by performing cots-ntttntion between the unified molecular representations and the learnable query tokens shown in Fig. 1 (Q- Former). worth noting that Q-Former is advantageous for modeling graphs, as the cross-attention guarantees the permutation invariance. We opt to initialize Q-Former with SciBERT [2].
Large Language Models choose Llama-2-7b-chan [46] and Llama-3. 13.1-8B-Inntruue [16], which have demonstrated their capabilities in the multi-moda. instruction tuning.
3.3 Training
Now, we turn out to introduce two training stages: molecular representation learning to train the projectors including the blending module and Q-Former, and end-to-end caatucction-tuinng
Molecular Representation Learning In the first stage, we train the blending module and the Q-Former while freezing the 2D and 3D encoders. We adopt the multi-objectives align the molecular embeddings to the molccule-relevant texts, including molecule-text contrastive learning, molecule-text matching, and molecule- grounded generation [24, 25]. We opt use the IUPAC geleuule-grounde
name as the molecule-relevar texts instead of using descriptions. Please refer to Section B.2 for detailed explanation of molecular representation learning.
End-to-end Instruction Tuning As shown in Fig. 1, we jointly train the blending module, Q- Former, and an L.LM via the multi-modal instruction tuning, while freezing the 2D and 3D encoders. We instruction-tune LLMs the proposed instruction dataset, employing LoRA [19] for the training efficiency. For the details of the instruction tuning of Mol-LLaMA, please refer Section B.2.
Experimental Results
We evaluate the capabilities of Mol-LLaMA by assessing the quality of generated responses to general questions and conducting the molecular property prediction and the molecular comprehension tasks.
4.1 Evaluation of General Understanding of Molecules
4.1.1
Qualitative Evaluation
To show how well Mol-LLaMA understands the molecular structures and their properties, we ask general question for molecule whose properties widely studied shown in Table 1. Interestingly,

---

Table 3: Quantitative evaluation the quality of generated responses for five criteria including helpfulness, relevance, accuracy, level detail and overall score. report the average relative score (i.e. score of an LLM divided by score of GPT-4o) by running GPT-44 evaluation three times. Molecularr LLMs that are trained on Llama3. 1-B- -8-IInttruc following their official implementations.
STRUCTURAL
CHEMICAL
Rioi OGGIACL
Models
Help.
Acc.
Details Overall
Help.
Relev
Acc.
Details Overall
Help.
Relev.
Details
Overall
LJma22-78 -BB-Bared
Jama2277-Chha
0.312
0.333
0.207
0.284
0.279
0.447
0.437
0.304
0,415
0.394
0.436
0.422
0.335
0.449
0.405
Mol-Innstuctions
0.218
0.249
0.210
0.144
0.207
0.250
0.280
0.254
0.168
0.235
0.351
0.448
0.425
0.253
0,360
LlasMel
0.251
0.266
0.221
0.192
0.228
0.273
0.301
0.235
0,213
0.252
0.346
0.410
0.390
0.298
0.353
3D.MoLM
0.550
0.541
0.426
0.542
0.507
0.669
0.666
0.557
0.661
0.628
0.836
0.894
0.855
0.892
0.862
LLaMo
0.314
0.396
0,348
0.206
0.310
0.359
0.459
0.447
0.240
0.361
0.498
0.734
0,803
0.340
0.568
Mol-LLaMA (Ours)
1.105
1,105
1.242
1,288
1.185
1.495
1.706
Llama3 or LlamaZ. J 1-B Bazed
11am331188
0.612
0.636
0.484
0.567
0.569
0.654
0.658
0.523
0.606
0.610
0.064
0.665
0.589
0.644
0.641
Mol-Instractions
0.257
0,315
0.282
0,166
0.253
0.274
0.359
0.322
0.179
0.276
0.392
0.547
0.355
0.259
0.423
3D-MoLM!
0.778
0.800
0.680
0.759
0.749
0.882
0.936
0.838
0.854
0.875
1.105
1.272
1.292
1.145
1.191
LLaMo
0.445
0.565
0.465
0.312
0.442
0.410
0.542
0.489
0.295
0.425
0.650
0.905
0.898
0.441
0.705
Mol-LLLMAA (Ours)
1.126
1.145
1.154
1.090
1.125
1.224
1.266
1.302
1.211
1.251
1.578
1.840
2,030
1.528
1.744
Mol-LLaMA accurately predicts the main class of the given molecule, explains the related properties, and provides the rationales for the predicted properties by learning the general knowledge from the proposed dataset. In contrast, GPT-4o and 3D-MoLM misinterpret the key structures, failing to provide correct properties, while LLaMo and For the additional qualitative results, please refer to Table 11, 12, and 13 in Appendix A.
do not explain the relevant properties.
4.1.2 Quantitative Evaluation
Experimental Setting We first select 100 representative unseen molecules from PubChem by conducting the k-means clustering based on Morgan Fingerprints [38]. Then, instruct to describe structural, chemical, or biological features, respectively. Please refer Appendix C.I for detailed explanations of experimental settings.
Evaluation Setting Inspired by Liu et al. [27], we leverage GPT-4o to measure the quality of generated responses. Specifically, after gathering the responses, we provide string representations and the original description as references for judging, and then instruct GPT-4o to assess the quality for the five criteria: helpfulness, relevance, accuracy, level of details, and the overall scores. We report the relative scores compared to GPT-4o (i.c. the score of an LLM divided by the score of GPT-4o). For detailed evaluation settings, please refer Appendix C.I.
Results As shown in Table 3, relative scores of Mol-LLaMA are beyond for all criteria, indicating that it is superior to GPT-4o in the understanding of general features of molecules, whereas the scores of other baselines are mostly lower than 1. On the other hand, Mol-LLaMA shows a significant performance improvement compared to the base LLMs and outperforms all baselines the same architecture, indicating that the proposed instruction dataset and blending module effectively expand the molecular knowledge and introduce the reasoning capabilities to Mol-LLaMA.
4.2 Molecular Property Prediction
Experimental Setting To assess the effectiveness of learning the general knowledge, we perform zero-shot evaluation on the PAMPA task [47]. The task is classifying the permeability of artificial membranes, requiring an understanding of essential molecular properties such as lipophilicity and molecular size. To evaluate the ability to handle diverse requests, we test on two additional prompting settings: 1) CoT [51] that instructs to provide rationales while answering and 2) prompting with task-specific information (w/ Task Info). Detailed evaluation settings are provided in Appendix C.2.
Evaluation Setting We evaluate on three metrics: accuracy, fidelity, and helpfulness. Fidelity and helpfulness measure the quality of responses regardless of whether the final prediction is correct or not, to assess the qualifications of a practical assistant. We leverage GPT-4o to evaluate the fidelity and helpfulness and report the relative score compared to GPT-40 (i.e. the score of an LLM divided by the score of GPT-4o). Additionally, we report the ratio of the predicted labels to check whether an LLM is biased to predict the labels. Please refer to Appendix C.2 for detailed prompts.
7

---

Table 4: Zero-shot performances PAMPA task. We highlight the best results in bold, except for the cases where predicted labels are identical which are denoted as N/A denotes the cases which more than 20% of the responses do not follow the answer format. Molecular LLMs that are trained on Alama3.1-88 8--Itttuuct
following their official implementations.
Default
CoT
w/ Task Info.
Models
Acc. (Ratio)
Fidel.
Help.
Aoc. (Ratio)
Fidel.
Help.
Acc. (Ratio)
Fidel.
Help.
GPT-4o
48.65 (39.95)
58.23 (47.42)
47.17 (62.41)
Llama2-7B 7B-Based
Llama2- 7B-Chat
57.14 (36.12)
0,517
0.508
57.53 (39.56)
0.639
0.658
84.52 (0.00)"
0.658
0.718
Mol-Instructions
49.63 (47.67)
0.277
0.210
31.16 (70.02)
0.314
0.270
38. (68.80)
0.331
0.256
LLaMo
84.28 (0.74)
0.242
0.187
84.52 (0.00)
0.246
0.191
N/A
0.226
0.185
Mol-LLaMA (Ours)
75.68 (11.30)
0.781
0.820
79.61 (6.88)
0.759
0,793
67.90 (28.75)
0.757
0.744
Llama3 Llama3. 1.8B-Boxed
Llama3.1-88 1-BB
56.51 (45.70)
0.629
0.554
46.19 (58.48)
0.795
0.786
63.64 (34.15)
0.850
0.875
Mol-Instractions
55.91 (38.33)
0.245
0.207
33.50 (73.96)
0.299
0.247
70.47 (25.55)
0.245
0.206
MD-Mol.Mt
46.93 (58.72)
0.668
0.651
50.00 (51.35)
0.671
0.649
64.86 (35.87)
0.767
0.744
ILaMo
49.25 (51,74)
0.265
0.212
64.37 (28.50)
0.254
0.209
48.51 (53.73)
0.401
0.327
Mol-LLaMA (Ours)
63.55 (36.86)
0.804
0.829
64.37 (31.94)
0.819
0.848
72.48 (17.44)
0.927
0.966
Result Table 4 shows that Mol-LLaMA achieves high accuracy outperforming GPT-4o, while showing high fidelity and helpfulness scores, demonstrating that it is able accurately predict the molecular property with helpful explanations. Further, compared to the base LLMs, Mol-LLaMA shows performance gain for both the accuracy and the quality of the responses by learning the general knowledge and reasoning ability from our constructed dataset. In contrast, other baselines lose the ability to provide relevant and helpful explanations, showing low fidelity and helpfulness scores compared to the base LLMs. Notably, Mol-LLaMA trained on Llama3.1 shows a consistent performance improvement when using CoT prompting and providing task-specific information, showing its ability to handle diverse requests from users. We note that the models whose relative scores of fidelity and helpfulness are below 0.3 fail to provide helpful explanations, as shown in Table 26 and 27. For detailed analysis of the generated responses, please refer to Section A.8. We further provide experimental results on another task (i.e. BBBP) in Table 10 Section A.2.
4.3 Molecular Comprehension Benchmark: Molecule
Experimental Settings To evaluate the fac- tual accuracy in the molecular comprehension, we employ MoleculeQA benchmark [33]. Af- ter fine-tuning the molecular LLMs, we report the accuracy of the answers to the questions. Please refer to Section C.3 for the introduction of MoleculeQA and the experimental details.
Table 5: Performance comparison on MoleculeQA.
Baseline results are taken from Lu et al. [33].
Model
Struct. Source
Prop.
App.
Totals
75-based
MolTS-small
49.59
64.18
46.51
40.90
51.69
MolTS-base
58.01
65.85
45.14
42.24
55.39
BioTS-base
65.98
69.24
49.11
40.73
62.03
MoMu-small
52.71
63.44
44.87
40.57
52.96
MoMu-base
61.58
65.30
43.78
43.07
57.43
Result As shown in Table 5, Mol-LLaMA con- sistently outperforms the other baselines on the same model architectures on all aspects, includ- ing structure, source, property, and application. Notably, Mol-LLaMA trained on Llama-3.1-88 architecture shows the best performance, indi- cating that the pre-trained knowledge of Mol- LLaMA and the high expressiveness from the proposed blending module contribute to compre- hending the molecular features and their behav- iors. We provide entire results in Table 19 and additional experimental results for task transfer on Mol-Innsrusttins [12] and 3D-MolT [25] in Table 17 and 18 Section A.6.
Galactica-hasdt MoICA-125M
65.54
67.34
45.77
40.33
60.30
MoICA-1.3B
71.12
70.98
47.81
43.17
64.79
Llama2- --- Based
Mol-Instruction
37.46
47.36
32.69
29.88
38.37
BioMedGPT-LM
54.19
60.01
40.90
52.23
3D-MoLM
69.64
68. 68.29
43.19
43.81
63.31
LLaMo
65.43
67.14
45.12
44.33
61.08
Mol-LLaMA
75.33
73.20
45.26
45.71
67.97
LJamad Llama3.1
1-88- Raxed
Mol-Instructions
75.93
73.96
46.22
44.36
68.45
3D-MoLM'
76.31
73.64
47.93
47.33
69.10
LLaMo
70.56
66.63
44.60
45.18
63.74
Mol-LLaMA
77.81
75.50
49.63
49,30
70.76
8

---

Table 6: Ablation studies (Left) data types and (Right) molecular representations. S denotes the detailed structural descriptions, S2F denotes the structure-to to-atturee relationship explanations, and Conv. denotes the comprehensive conversations. We report overall scores following the settings in Section 4.1.2. For the ablation study on data types, we report the accuracy on PAMPA task.
STRUCT.
CHEM.
BIO.
PAMPA
STrUCT.
CHEM.
BIO.
Models
Overall
Overall
Overall
Default
CoT
w/ Task Info.
Models
Overall
Overall
Overall
S
1.119
1.166
1.328
37.35
32.92
56.02
2D
0,907
1.137
1.526
S+S2F
1.122
1.285
1.754
66.83
64.13
57.88
3D
1.071
1.195
1.632
Conv.
1.166
0.689
0.887
41.03
45.95
83.05
2D+3D (Concat)
1.037
1.210
1.741
Full
1.125
1,251
1.744
63.55
64.37
72.48
2D+3D (Blended)
1.125
1,251
1.744
2D+3D (Blended): The given molecule, diphenyl ether, consists two phenyl rings connected by an oxygen atom, forming an ether linkage. ... The oxygen atom is boned to both phenyl rings, making it an ether. This structure results in a symmetrical molecule with planar geometry around the ether linkage.
Diphenyl Fther Input Molecule (GT)
2D+3D (Concat): The given molecule, 1.4-diphenoxybenzeee consists of a benzene ring substituted with two phenoxy groups Each phenoxy group is attached to the benzene ring at opposite positions, specifically at the I and 4 positions, which are para to each other. This means that the two phenoxy groups are located on the benzene ring a symmetrical manner. Creating a para substitutt structure.
Lddipphenoybeeeee Predicted Structure
3D: The given molecule, 1-phenylbenzene, is a type of aromatic compound known as a benzene derivative. Its structure consists of two benzene rings connected by single covalent bond. ... The presence of two benzene rings makes the molecule relatively planar, allowing for potential stacking interactions and contributing to its physical properties, such as its crystalline structure.
1-phenylbeneene Predicted Structure
2D: The given molecule, phenylphosphino. consists of a phosphorus atom bonded to phenyl group. The phenyl group is an aromatic ring, specifically benzene ring, which six-carbon ring with alternating double bonds. The phosphorus atom directly attached the henzene ring, forming a phosphine linkage....
pheno//phoophiiee Predicted Structure
Table 7: Examples of structural descriptions for Diphenyl Ether with different molecular representation types.
4.4 Ablation Study
Data Type We ablate the different data types in Mol-LLaMA-Instiee to show the effect of each data type. As shown in Table 6 (Left), learning structural descriptions exclusively or with structure- to-feature relationship explanations helps understand the general features. On the other hand, even though learning comprehensive conversations shows an inferior understanding of chemical and biological features, allows to handle the diverse contexts of users' requests showing a large performance gain when predicting molecular property with task-specific information. Training on the full data balances this trade-off, showing moderate performances both on the understanding of general features and the ability to handle the users' inquiries. Please refer Table 22 of Appendix A.7 for the entire scores including helpfulness, relevance, accuracy, and level of detail.
Blending Module To show the effectiveness of the proposed blending module, we ablate the different types of molecular representations. As shown in Table 6 (Right), using single type of encoder (2D and 3D) still shows a superior understanding of molecular modality compared to GPT-4o, thanks to the proposed instruction dataset. Concatenating 2D and 3D representations without the blending module (2D+3D (Concat)) results in the degeneration of the structural understanding. Using the blending module (2D+3D (Blended)) outperforms other variants, indicating that integrating the complementary information from different molecular representations is crucial to enhance the understanding of the structures and the advanced features. Qualitatively, as shown in Table 7, using a single encoder (2D or 3D) struggles to correctly predict the structural features, missing atoms or functional groups. On the other hand, while concatenating the molecular representations (2D+3D (Concat)) correctly predicts the presence of atoms and functional groups, it fails to predict the connectivity. The blending module (2D+3D (Blended) accurately predicts the molecular structures, showing the effectiveness of integrating the complementary information from different encoders.

---

Table 8: Quantitative evaluation with three diverse conformations generated by RDKit and OpenBabel.
STRUCTURAL
CHEMICAL
BIOLOGICAL
Help.
Reley.
Details
Overall
Help.
Relev.
Details
Overall
Help.
Relev.
Details
Overall
Fixed
1.127 1.145
1.154
1,090
1.125
1.224
1.266
1.302
1,211
1.251
1.578
1.840
2.030
1.528
1.744
Diverse
1.058
1,070
L1112
1,010
1,065
1,171
1.225
1,301
1,138
1.213
1.641
1,895
2.203
1,577
1.841
4.5 Further Analysis
Robustness Diverse Conformations To demonstrate robustness to conformational diversity, we evaluate Mol-LLaMA three different conformations for each representative molecule from Section 4.1.2. We note that these molecules have, on average, 11.11 rotatable bonds, ensuring diverse generated conformations. As shown in Table 8, performance with diverse conformations (Diverse) is largely consistent with that obtained using a single fixed conformation (Fixed) overall, indicating that Mol-LLaMA is robust to conformational variability.
Few-Shot Prompting with Multiple Molecules To evaluate the capabilities of processing and understand- ing multiple molecules, we apply a few-shot prompting method on the PAMPA task. As shown in Table 9, Mol- LLaMA attains a consistent performance improvement as the number of in-context examples increases, demon- strating that it effectively leverages multiple molecules and their labels. In contrast, GPT-44 fails to capitalize on additional examples; its performance declines when provided with one or three in-context exam- ples. Across all evaluated settings, Mol-LLaMA outperforms GPT-4o, reflecting the advantages of Mol-LLaMA from domain-specializzd modeling and structure-aware reasoning.
Table 9: Accuracy with few-shot prompting, which provides in-context examples of input- output pairs, on the PAMPA task.
Models
O-shot
1-shot
3-shot
5-shot
GPT-4o
48.65
31.45
46.68
57.74
Mol-LLaMA
63.55
73.46
77.64
80.10
Conclusion
In this work, we present Mol-LLaMA, a large molecular language model with a general understanding of molecular features and reasoning capabilities. To this end, we present a large instruction dataset by devising three data types, to cover the fundamental features from structural features chemical and biological features and cultivate the explainability and reasoning capabilities. Further, we propose the blending module capture the structural information from different encoder types, enhancing the understanding of structures and advanced properties. Experimentally, we show that Mol-LLaMA is capable of not only predicting molecular features accurately but also providing informative responses, implying its utility as ppurpss assistant for molecular analysis. We hope that our dataset construction pipeline and proposed model architecture can utilized for other modalities in the scientific fields such as proteins, RNAs, and their complexes. Further, we hope that Mol-LLaMA can be used to solve scientific problems based on its capabilities as a general-purpos. molecular assistant.
Acknowledgement This work was supported by Institute for Information & communications Technology Planning & (valuation(IITPP) grant funded by the Korea government(MSIT) (RS-2019- 11190075, Artificial Intelligence Graduate School Program(KAISTT Learning Ledger Orchestration for Drug Discovery Project( (MMLLLOOK- funded by the Ministry of Health & Welfare and Ministry of Science and ICT, Republic of Korea (grant number: RS-2024- 00460870), Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT(MSIT, Korea) & Gwangju Metropolitan City, National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-00256259) Institute of Information & communications Technology Planning & Evaluation grant funded (No.RS-202) W-R-SS222-I122220 11220113, Meta-learning Applicable
a grant of the Korea Machine
by the Korea governmen(MMSI) Problems), the Institute of Information & Communications Technology Plamning & Evaluation (IITP) with grant funded by the Ministry Science and ICT (MSIT) the Republic of Korea connection with the Global AI Frontier Lab International Collaborative Research (No. RS-2024-00469482 & RS- 2024-00509279 & RS-2024-00509255), and Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (RS-2024-0041475/)
10

---

References
[1] Microsoft Research A14Science and Microsoft Azure Quantum. The impact of large language models on scientific discovery: a preliminary study using gpt-4. arXiv:2311.0736/ 2023.
[2] Iz Beltagy, Kyle Lo, and Arman Cohan. Scibert: A pretrained language model for scientific text. arXiv preprint 1eXXvv1903.106777 2019.
[3] He Cao, Zijing Liu, Xingyu Lu, Yuan Yao, and Yu Li. Instructmol: Multi-modal integration for building versatile and reliable molecular assistant in drug discovery. (.Xiv:23/1./6200 2023.
[4] Juan Manuel Zambrano Chaves, Eric Wang, Tao Tu, Eeshit Dhaval Vaishnav, Byron Lec. Sara Mahdavi, Christopher Semturs, David Fleet, Vivek Natarajan, and Shekoofch Azizi. Tx-llm: A large language model for therapeutics. arXiv:2406.0633 2024.
[5] Guo Chen, Yin-Dong Zheng, Jiahao Wang, Jilan Xu, Yifei Huang, Junting Pan, Yi Wang, Yali Wang, Yu Qiao, Tong Lu, et al. Videollm: Modeling video sequence with large language models. arXiv:2305. 2023.
[6] Seyone Chithramanda, Gabriel Grand, and Bharath Ramsundar. Chemberta: large-scale self- supervised pretraining for molecular property prediction. arXiv:2010.09883 2020.
[7] Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, and Mat- teo Manica. Unifying molecular and textual representations via multi-task language modelling. In International Conference Machine Learning, 2023.
[8] Gheorghe Comanici et Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities. arXiv:2507.062611 2025.
[9] Oscar Dall' Igna, Lisiane O Porciúncula, Diogo O Souza, Rodrigo A Cunha, and Diogo Lara. Neuroprotection by caffeine and adenosine a2a receptor blockade of 3-amyloid neurotoxicity. British journal of pharmacology, 138(7):1007-1209, 2003.
[10] Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. In Proceedings of the 2022 Conference Empirical Methods in Natural Laggagee Processing, 2022.
[11] Benedek Fabian, Thomas Edlich, Héléna Gaspar, Marwin Segler. Joshua Meyers, Marco Fiscato, and Mohamed Ahmed. Molecular representation learning with language models and domain-relevant auxiliary tasks. arXiv:2011.13230, 2020.
[12] Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. In International Conference on Learning Representations, 2024.
[13] Yassir Fathullah, Chunyang Wu, Egor Lakomkin, Ke Li, Junteng Jia, Yuan Shangguan, Jay Mahadeokar, Ozlem Kalinli, Christian Fuegen, and Mike Seltzer. Audiochatllama: Towards general-purpose speech abilities for lms. In Association for Computational Linguistics: Human Language Technologies (Volume Long Papers), 2024.
[14] Henri A Favre and Warren H Powell. Nomenclature of organic chemistry: IUPAC recommenda- tions and preferred names 2013. Royal Society of Chemistry, 2013.
[15) Xiang Fu, Brandon M. Wood, Luis Barroso-Luque, Daniel S. Levine, Meng Gao, Misko Dzamba, and C. Lawrence Zitnick. Learning smooth and expressive interatomic potentials for physical property prediction. 25X.:22502.12147 2025.
[16] Aaron Grattafiori et al. The llama 3 herd of models. arXiv:2407.21783, 2024.
[17] Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1-23, 2021.

---

[18] Lesley Heseltine, Judith M Webster, and Roy Taylor. Adenosine effects upon insulin action on lipolysis and glucose transport in human adipocytes. Molecular and cellular biochemistry, 144: 147-151, 1995.
[19] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Yuanzhi Li. Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. Internation Conference on Learning Representations, 2021.
[20] Ross Irwin, Spyridon Dimitriadis, Jiazhen He, and Esben Jannik Bjerrum. Chemformer: a pre- trained transformer for computational chemistry. Machine Learning: Science and Technology, 3(1 31))01505 2022.
[21] Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, et al. Pubchem in 2021: new data content and improved web interfaces. Nucleic acids research, 49(D1):D1388 D1395, 2021.
[22] Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alan Aspuru-Guzik. Self-referencing embedded strings (selfies): 100% robust molecular string representation. Machine Learning: Science and Technology, 144):04522,, 2020.
[23] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234-12)) 1240, 2020.
[24] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. International Conference on Machine Learning, 2023.
[25] Sihang Li, Zhiyuan Liu, Yanchen Luo, Xiang Wang. Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, and Qi Tian. 3d-molm: Towards 3d molecule-text interpretation in language models. In International Conference on Learning Representations, 2024.
[26] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. Conference on Computer Vision and Puttern Recognition, 2024.
[27] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in Neural Information Processing Systems, 2024.
[28] Pengfei Liu, Yiming Ren, Jun Tao, and Zhixiang Ren. Git-mol: A multi-modde large language model for molecular science with graph, image, and text. Computers in biology and medicine, 2024.
[29] Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar. Mudai molecule structure text model for text-based retrieval and editing. Nature Machine Intelligence, 5(12): 144--1457, 2023.
[30] Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. MolCA: Molecular uni-modal adapter. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, 2023.
modeling with modal projector and
[31] I Loshchilov. Decoupled weight decay regularization. International Conference on Learning Representations, 2019.
[32] Shuqi Lu, Zhifeng Gao, Di He, Linfeng Zhang, and Guolin Ke. Data-driven quantum chemical property prediction leveraging 3d conformations with uni-mol+. Nature Communications, 15 (1):7104, 2024.
[33] Xingyu Lu, He Cao, Zijing Liu, Shengyuan Bai, Leqing Chen, Yuan Yao, Hai-Tao Zheng, and Yu Li. MoleculeQA: A dataset to evaluate factual accuracy in molecular comprehension. In Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics, 2024.
12

---

[34] Yizhen Luo, Kai Yang, Massimo Hong, Xing Yi Liu, and Zaiqing Nie. Molfm: A multimodal molecular foundation model. arXiv preprint XXXi:2337700984 2023.
[35] Yizhen Luo, Jiahuan Zhang, Siqi Fan, Kai Yang, Massimo Hong, Yushuai Wu, Mu Qiao, and Zaiqing Nie. Biomedgpt: An open multimodal large language model for biomedicine. IEEE Journal of Biomedical and Health Informatics, 2024.
[36] Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), 2024.
[37] Oscar Méndez-Lucio, Christos A Nicolaou, and Berton Earnshaw. Mole: foundation model for molecular graphs using disentangled attention. Nature Communications, 15(1):9431, 2024.
[38] Harry L Morgan. The generation of unique machine description for chemical structures-a technique developed at chemical abstracts service. Journal of chemical documentation, 5(2): 107-113, 1965.
[39] OpenAl. Gpt-4 technical report. arXin:2303.08774, 2024.
[40] OpenAl. Gpt-4o system card. arXiv:2410.21276, 2024.
[41] Jinyoung Park, Minseong Bae, Dohwan Ko, and Hyunwoo J Kim. Llamo: Large language model-based molecular graph assistant. In Advances in Neural Information Processing Systems, 2024.
[42] Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingee Xia, and Rui Yan. Biot5: Emriching cross- modal integration in biology with chemical knowledge and natural language associations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2023.
[43] Shaghayegh Sadeghi, Alan Bui, Ali Forooghi, Jianguo Lu, and Alioune Ngom. Can large language models understand molecules? BMC bioinformatics, 25(1):225, 2024.
[44] George Serhan, Colin M Stack, Gabriel G Perrone, and Charles Oliver Morton. The polyene antifungals. amphotericin b and nystatin, cause cell death in saccharomyces cerevisiae by distinct mechanism amphibian-derivee antimicrobial peptides. Annals of clinical microbiology and antimicrobials, 13:1-4, 2014.
[45] Bing Su, Dazhao Du, Zhao Yang, Yujie Zhou, Jiangmeng Li, Anyi Rao, Hao Sun, Zhiwu Lu, and Ji-Rong Wen. molecular multimodal foundation model associating molecule graphs with natural language. arXiv:2209.0544
2022.
[46] Hugo Touvron et al. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288, 2023.
[47] Alejandro Velez-Arce, Xiang Lin, Michelle Li, Kexin Huang, Wenhao Gao, Tianfan Fu, Bradley L. Pentelute, Manolis Kellis, and Marinka Zitnik. Signals in the cells: Multimodal and contextualized machine learning foundations for therapeutics. Advances in Neural Information Processing Systems Workshop Al for New Drug Modalities, 2024.
[48] Eric Wang, Samuel Schmidgall, Paul F. Jaeger, Fan Zhang, Rory Pilgrim, Yossi Matias, Joelle Barral, David Fleet, and Shekoofeh Azizi. Txgemma: Efficient and agentic 1lms for therapeutics. arXiv:2504.061 006.200040666..... 2025.
[49] Longsheng Wang. Jie Zhang, Wenxin Zhang, Mingming Zheng, Hongjie Guo, Xiaohui Pan, Wen Li, Bo Yang, and Ling Ding. The inhibitory effect of adenosine on tumor adaptive immunity 2024.
and intervention strategies. Acta Pharmaceutica Sinica B,
[50] Sheng Wang, Yuzhi Guo, Yuhong Wang, Hongmao Sun, and Junzhou Huang. Smiles-bert large scale unsupervised pre-training for molecular property prediction. In Proceedings of the 10th ACM international conference on bioinformatics, computational biology and health informatics, pages 429-436, 2019.
13

---

[51] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. ohain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 2022.
[52] David Weininger. Smiles, chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28 (1):31-36, 1988.
[53] Brandon M. Wood, Misko Dzamba, Xiang Fu, Meng Gao, Muhammed Shuaibi, Luis Barroso- Luque, Kareem Abdelmaqsoud, Vahe Gharakhanyan, John R. Kitchin, Daniel S. Levine, Kyle Michel, Anuroop Sriram, Taco Cohen, Abhishek Das, Ammar Rizvi, Sushree Jagriti Sahoo, Zachary W. Ulissi, and C. Lawrence Zitnick. Uma: A family of universal models for atoms. arXiv:2506.23971 2025.
[54] Di Wu, Qi Chen, Xiaojie Chen, Feng Han, Zhong Chen, and Yi Wang. The flood-brait barrier: structure, regulation, and drug delivery. Signal Transduction and Targeted Therapy, 8(1):217, 2023.
[55] Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. arXiv:2411.104400 2024.
[56] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Yan Liu. Do transformers really perform badly for graph representation? In Advances in Neural Information Processing Systems, 2021.
[57] Botao Yu, Frazier N. Baker, Ziqi Chen, Xia Ning, and Huan Sun. Llasmol: Advancing large language models for chemistry with large-scale, comprehensive, high-quality instruction tuning dataset. arXiv:2402.09391 2024.
[58] Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in Ilms beyond the base model?, 2025.
[59] Zheni Zeng, Yuan Yao, Zhiyuan Liu, and Maosong Sun. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals. Nature communications, 13(1):862, 2022.
[60] Hang Zhang, Xin Li, and Lidong Bing. Video-LLaMA: An instruction-tuned audio-visual language model for video understanding. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations 2023.
[61] Juzheng Zhang, Yatao Bian, Yongqiang Chen, and Quanming Yao. Unimot: Unified molecule- text language model with discrete token representation. arXiv:2408.0085 2024.
[62] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595-46622 2023.
[63] Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In International Conference on Learning Representations, 2023.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
15

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
There is no theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: Set////oodumiiic
/ //iiss... //nips.cc/puit wttpccccccpppcccccccceocococpcce
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting//etails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer:
[Yes]
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: On the test-time, we use greedy decoding without randomness. Additionally, training LLMs multiple times is computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-simma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
18

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //pppr////uurps.../ wwtpuurpsscccppppeeeeee httpplc//tthissuiiieeee.poppppee
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
19

---

Released models that have high risk for misuse or dual use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
20

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https cttpr/pssseeeeoooeeneneeeeeeeeee hetpeencccee//////////// for what should or should not be described.
21

---

Appendix
Organization Appendix is organized as follows: In Section A, we provide additional results including qualitative comparison, molecular property prediction, task transfer, response analysis on PAMPA task, and ablation study. In Section B, we explain details of the instruction dataset construction and the training. In Section C, we provide experimental details for the evaluation settings. In Section D, we discuss the limitations and the societal impacts of our work.
A
Additional Experimental Results
A.1 Qualitative Results
In this section, we provide additional qualitative results to analyze the behaviors of Mol-LLaMA. First, we provide the entire responses of the case study in Table 1. As shown in Table 11, Mol-LLaMA understands the molecular structures and correctly predicts the main class of the given molecule (i.e. Benzodiazepine), the attached functional group (i.e. pyridine), and its biological functionalities. Interestingly, Mol-LLaMA provides additional information, detailing the effects of each structural component, such as the blood-brain barrier penetration from the pyridine ring, the binding affinity to the GABA-A receptor from the halogen atom, and the inhibitory effects of benzodiazepines. In contrast, GPT-4o, 3D-MoLM, and Mol-Instructions misinterpret the molecular structures and hence, mispredict incorrect features of molecules. On the other hand, I.I LaMo misinterprets the substructures (i.e. methyl group) and does not provide the distinct features of the given molecule.
Additionally, as shown in Table 12, one notable ability of Mol-LLaMA that it provides detailed rationales for the predicted properties. Specifically, Mol-LLaMA explains the key structure (i.e., macrolides) and additional structural features (i.e., polyene chain), and then explains that these structural features are related the specific biological properties (i.e., antifungal activity), providing principles of the predicted properties (i.e.. binding to ergosterol) [44]. However, due to its limitation in interpreting string representations, GPT-4o misinterprets molecular structures and consequently predicts unrelated properties. 3D-MoLM mispredicts the molecular properties, while LLaMo provide the source of this molecule, which is irrelevant the user's question.
Further, as shown in Table 13, beyond the properties annotated in PubChem, Mol-LLaMA understands various aspects of the molecular properties that can be found in the scientific literature, such as regulating the secretion of insulin [18], inhibiting the immune cells [49], and protecting neurons [9]. Similarily, GPT-4o provides relevant properties, correctly predicting the molecule name. However, LLaMo does not faithfully address the query, providing a simple treatment, not the biological functionalities.
A.2 Molecular Property Prediction on BBBP
We additionally conduct an experiment on BBBP task, where the task is to predict whether given molecule can penetrate the blood-brain barrier. The brain-blood barrier is system with high selectivity by complicated transport routes [54], requiring exter- nal knowledge such the structure of tight junctions, the interactions with membrane proteins, and the envi- ronment of BBB. We follow the experimental settings in Section 4.1.2, using three types of prompt strate- gies: default setting, CoT prompting, and providing task-specific information, As shown in Table 10, Mol- LLaMA shows high performance, achieving a per- formance improvement compared to the base LLMs. However, GPT-4o outperforms all baselines due its extensive knowledge.
Table 10: Zero-shot performances on BBBP task. We report accuracy for the test set fol- lowing the splitting from Zhou et al. [63]. The best results are highlighted in bold, and the recond-bess results are underlined.
Models
LIM
Defuslt
GPT-4o
60.82
61.34
64.43
Llama2
Llama2-78
37.37
51.56
$3.09
Liuma3
Llama3 1-88
57.07
51.03
55.15
Mol--nstmerionn
1 1ama2.78
52.58
$2.58
$1.34
Mol-Instuccioos
Llama3-66
LhsMMl
Mistral-BB
$1.55
54.64
$6.19
MDLM
Llam33.1-5
49.14
51.65
51.91
LLaMo
Llama2-7
49.36
50.60
LLaMe'
Llama3. 1-8
55.44
55.45
56,91
Ma (Ours)
Lama2-77
53.37
52.58
$2.58
Mol-LLaMA (Ours)
Llama3.1-80
1-88
39.54
53.56
59.08
22

---

Diazepine
Benzodiazepine
Quinzzoline
Quinoxaline
Ingur Molcesie
SMILES; CIGCOONCCECEEEGGG////CCCCCYCCCCC
Original Description: Bromazepam binds to the GABA receptor GABA-A, causing a conformational change and increasing inhibitory effects of GABA. Other neurotransmitters are not influenced. One of the benzodizzepines that is used in the treatment of anxiety disorders.
User: Explain the features of the given molecule that are most distinct from those of other molecules.
Mol-LLaMA (Ours)
The given molecule is a member of the benzodiazepine class, which is characterized by a fusion of a benzene ring and a seven- membered diazepine ring. This structure crucial for biological activity. particularly role sedative and hy hypnotic agent. One of the key features of the given molecule is the presence a pyridine ring attached benzodiazepine core. This pyridine moiety is significant because can influence the molecule's 's pharaackinnetic properties, such its ability cross the blood- barrier, which is essential for its function as central nervous system depressant.
Additionally, the molecule contains hromine atom attached to the henzodiazepine ring. The presence of this halogen atom can affect the molocule's 's lipophilicity and its binding affinity to the GABA A receptor, which the primary target of benzodiazepines. The bromine atom may also contribute to the molecule's metaholic stability and its duration action.
The given molecule also characterized by a ketone group within diazepine ring, which is a common feature benzodiazepines. This functional group important for the interaction with the GABA-A receptor, it plays a role in the formation of the benzodiazepine- mmwziiaazpine-recettoo complex, enhancing the inhibitory effects of GABA.
Overall, the structural features of the given molecule, inc including the henzodiazepine core, the nyridine ring. the hromine atom, and the ketone group, collectively contribute to its pharmacological properties as sedutive and hypnotic agent.
GPT-4o
The given molecule 6 6-bromo-3 3- (pyridin y1) IH quinazoline 2.4-ei.on. Several distinctive features set this molecule apart:
1. "*Quinazoline Core**: The molecule contains quinazoline core, which bicyclic structure combining a benzene ring and pyrimidine ring. This is distinct from simpler aromatic compounds due to the additional nimmgen atoms in the pyrimidine portion.
2. **Substittioon Pattern*** The molecule has unique substitution on the quinazoline core:
bromine atom substituted the 6 posoition of the quinazoline ring. This halogenation distinctive and can affect molecule's 's reactivity and hinding properties significantly, compared to halogenated quinazolines.
pyridin-2-y 2-yl group is attached at the 3- 3-position. This makes the compound heteroorylated quinazoline, which modulate the electromic properties and polarity of the molecule.
3. **Presence of Two Carbonyl Groups**; The molecule has two carbonyl groups at the 2-position and 4-position of the quinazoline ring. These functional groups contribute to potential hydrogen bonding and polarity, influencing the molecule's s chemical reuctivity und interuction with biological targets.
4. "HHeterouromaaic Ring Systems** Besides quinazoline care, the presence of a pyridine ring at 3-position adds second beteroaromatic system, increasing the overall aromatic character and introducing additional nitrogen atoms. which can serve as potential sites for interaction or coordination with metal ions.
Overall, the combination of quinazoline structure with unique substitution patterns, including bromine halogenation and the presence of pyridinyl group. defines the molecule's distinct chemical charucteristics. This structure is particularly relevant in medicinal chemistry, where such frameworks are often explored for developing biouctive compounds.
LLaMo
The molecule has a benzodiazepine structure with a bromo substituent at the 7-position and a methyl group at the I-position. It is distinct from other molecules that it contains bromine atom, nitrogen atom, and methyl group, which are not present other molecules.
3D-MoLM
The molecule with the SMILESS code CICC0))NN-=<(CC(((CC)))CCCCCCE CCCCB//CCCCCCCCCCCCCCCCCCCCCCCCC is known as 7- T-bromohypnotic. It belongs class of compounds called quinazolines, which are characterized by bicyelic stucture consisting benzene ring fused pyrimidine ring. The molecule contains carbonyl group (C=O) and an amide group (C(=O)N) attached to the pyrimidine ring. These functional groups contribute to the molecule's 's ability to form hydrogen bonds and participate in various chemical reactions. The presence of bromine atom (Br) on benzene ring imparts unique properties to 7-bromohyppotti Bromine is a halogen that is more electmnegatiiv than carbon, resulting in polar hond. This polarity can influence the molecule's 's reactivity and solubility in different solvents. Additionally, the molecule contains a nitrogen atom (N) in the pyrimidine ring. which can
basic center and participate acid-base reactions. The presence conjugated double bonds the benzene and pyrimidine rings gives rise to delocalized pi-clectron systems, leuding to enhanced stability and potential for aromaticity. Overall, 7-bromohypnede pontocypooiicccccooo exhibits distinct features such as the presence bromine atom, curbonyl and amide groups. and conjugated double bonds, which contribute to its unique chemical and physical properties.
The molecule's structure suggests potential applications medicinal chemistry, quinazolines have been found to possess various biological activities, including untitumor, untimicrobial, und unti-inflammatory properties. Additionally, the presence of a hromine atom can enhance the molecule's x lipophilicity, potentially affecting its bioavailahility and distribution in the body. Further studies and experiments necessary to fully understand the specific biological activities and potential therapeutic of 7- broomhhynootcc. However, its unique molecular features make it an intriguing compound for lurrherr investigation in the field of chemistry and biology.
Mol- Instructiooss
The molecule quinoxaline derivative.
Table 11: Entire responses for the case study in Table 1.
23

---

Original Description from PubChem
Nystatin is a polyene antifungal drug that has broad-spectrum fungicidal and fungistatic activity against number of yeasts and fungi. most notably Candida species. Nystatin is polyene macrolide antibiotic, with antifungal activity. Upen administration systatin binds to sterols in the fungal plasma membrone, thereby increasing membrane perneability. This leads to leakage of important intracellular components, cell rupture, and eventually fungal cell death.
Nystalin
User: Could you explain the biological implication of this molecule?
Mol-LLLaMA
The given molecule is a complex natural product known for its potent antifungal properties. is produced by the fungus *Streptomyces nodosus and is classified as a polyene macrolide. The hiological implications of this molecule are primarily related to its ability to disrupt
cell membranes fungi, making it an effective antifungal agent.
The structure of the given molecule includes a large lactone ring, which characteristic macrolides. This ring conjugated with multiple conjugated double honds, forming a polyene chain. The polyene chain is crucial for its antifungal activity, it allows the molecule to bind to ergosterol, a key component of fungal cell membranes. This binding results in the formation of and molecules, ultimately causing cell death.
pores in the membrane, leading to the leakage of essential
Additionally, the presence of multiple hydroxyl groups and an amino sugar moiety in the structure enhances the molecule's s solubility and binding affinity the fungal cell membrane. The curboxylic acid group contributes the molecule's 's overall polarity and may play mole in its interaction with the fungal cell surface.
Overall, the given molecule's 's structure is intricately linked to its biological function antifungal agent, making it valuable compound in the treatment of fungul infections.
GPT--0
The molecule you' provided is Retinyl j-D-glucoside, Here's a brief explanation of its biological implications:
Wel Retinyl B-D B-D-glucoside
1. *SStructure and Composition **:
Retinyl d-D-glucoside is a derivative of vitamin A (retinol) linked to glucose molecule.
The retinoid structure is vital for its function, influencing various biologrical processes.
2. *** "Biological Ro
*Viisio***: As form of Vitumin A, plays crucial role in the visual cycle. Retinoids are essential for the formation of retinal, a molecule required for the conversion of light into visual signals in the retina.
- **Ceellular Differentiation** Retinoids are involved in the regulation of gene expression. They influence the growth and differentiation of epithelial cells, which is criticul for maintuining healthy skin and mucous membranes.
[mmune Function**, +*; Vitamin A derivatives support the immune system by enhancing the macosal barrier and facilitating the function of immune cells.
Antioxidant Properties**; Retinnids can protect cells from oxidative stress, which helps to prevent chronic diseases and certain degenerative conditions.
3. "Meetaolism and Function**:
Once ingested, retinyl (8- (---gllcooie D-gglucoside would likely be metabolized to retinal and then to other active retinoids like retinal and retinoic acid exert its physiological effects.
TTee presence the glucose moiety may affect its absorption, distribution, and biouvailability in the body.
Overall, the biological implications of this molecule are extensive due to its role as a vitumin A derivative, impacting vision, cellular health, immune function, and more.
LLaMo
The molecule is natural product found in Streptomyces with data available.
3D-MoLM
The molecule with the given SMILES code complex natural product found in Streptomyces ambofaciens. It is a polyketide compound with highly conjugated system of double bonds, giving it long and flexible structure. The molecule contains multiple functional groups, including hydroxyl (-OH) and carbonyl (C=0) groups, which contribute to its reactivity and potential biological activity. The molecule oxygen atoms, which are likely involved in hydrogen bonding and other interactions with biological targets. The molecule also contains sugar moiety, specifically a 6 6-amino-,,4 14,5-trihydmmxxe trihydroxoxxane group, which is attached to the central core. This sugar group important for the molecule's solubility and potential interactions with biological receptors. The presence of multiple hydroxyl groups suggests that this molecule may have antioxidant properties and could potentially interact with enzymes proteins involved in oxidative stress. Additionally, the presence of an amino group indicates that this molecule may have the potential to act as a base or form hydrogen bonds with other molecules. Overall, this molecule's 's complex structure and functional suggest that may have wide range of biological activities, including potential antioxidant, enzyme inhibition, or receptor binding properties. Further studies are needed to fully understand its specific biological implications and potential applications.
cyclic structure, with central core consisting of 22 membered ring. This ring contains several
The molecule's 's structure and functional groups suggest that it may have wide runge of biological activities, including potential antioxidant. enzyme inhibition, or receptor binding propertics. The presence of multiple hydroxyl groups indicates that may have antioxidant properties and could potentially interact with enzymes or protens involved in oxidative stress. The molecule's 's complex structure and the presence of sugar moiety suggest that it may have the potential to interact with biological receptors or enzymes. Further studies are needed to fully understand its specific biological implications and potential applications. The molecule's 's natural origin in Streptonyces ambofaciens suggests that it may have evolved to play a role in the organism's is survival or defense mechanisms. Further investigation into its biosynthesis and coological role may provide insights into its potential applications in medicine or agriculture. In summary. this molecule's 'ss complex structure and functional groups make promising candidate for further investigation into its biologicul activities and potential applications. Its natural origin and potential interactions with biological targets make valuable compound agriculture. Overall, this molecule's 's unique structure and functional groups make it a fascinating compound with potential for various hiological activities. Its natural origin and potential applications make it an important target for further investigation and potential development in the fields of medicine and agriculture. Further studies are needed to fully understand its specific biological implications and potential applications.
further research and potential development in the fields medicine and
Table 12: Case study for Nystatin. Mol-LLaMA correctly understands the molecular features and explains them in detail.
24

---

Original Description from PubChem
Adenosine is nucleoside that is composed of adenine and d-ribose. Adenosine adenosine derivatives play many important biological roles in addition to being components of DNA and RNA. For instance, adenosine plays an important mlee in energy transfer as adenosine triphos- phate (ATP) and udenosine diphosphate (ADP). also plays role signal transduction as eyelic adenosine monophoxphate, cAMP, Adenosine itself is both neurotransmitte and potent vasodilator. When administered intravenously, adenosine causes transient heart block in the node. Because of the effects adenosine on AV node-dependent supraventricular tachycardia, adenosine is considered a class V antiarhyytmic agent.
dn
Adenssine
User: What the biological functionalities of this molecule?
Mol-LLaMA
The given molecule, commonly known as adenosine, is a purine ribonucleoside composed of two main components: an adenine base and a ribose sugur. This structure allows it to play several critical biologicul roles:
I. ""Energy Transfer and Storage**: Adenosine is a precursor to adenosine triphosphate (ATP), which is the primary energy eurrency of the cell. It is involved the storage and transfer of energy in cellular processes.
2. *"Signal Transduction th; As signaling molecule, adenosine acts as a neuromodulater in the central and peripheral nervous systems. It hinds specific receptors, such A1, A2A, A2B, and A3, influencing various physiological processes, including vasodilation, auti-inflammatorry responses, and modulation of neurotransmitter release.
3. Mettboolic Regulation Adenosine is involved in the regulation of glucose and lipid metabolism. It affects insulin secretion and sensitivity, which crucial for maintaining glacose homeostasis.
4. **Candovascullar *;nction**; In the cardiovascular system, adenosine plays a role in regulating heart rate and blood pressure. It acts as a vasodilator, which helps in reducing blood pressure and improving blood flow.
5. "*Immune System Modulation"*; Adenosine can modulate immune responses by inhibiting the activity of immune cells, such as lymphocytes and macrophages. This helps preventing excessive inflammation.
6. *NNuroprotcction **; Adenosine has neuroprotective properties, which may help in protecting neurons from damage and degeneration. It is involved in the regulation of synaptic plasticity and neuronal survival.
7. **Role DNA and RNA** As mucleoside, adenosine building block of nueleic acids, such as DNA and RNA. It is essential for the synthesis and repair of genetic material.
8. **Cellular Communication Adenosine is involved in cellular communication processes, such the regulation of cell growth, differentiation, and apoptosis.
Overall, the given molecule is a multifunctional compound that plays vital role in various physiological and biochemical processes in the body.
GPT-40
The molecule you provided is udenosine, nucleoside composed of adenine attached ribose sugur. Here are its biologicul functionalities:
I. *Commoonntt of Nucleic Acids"*: Adenosine is fundamental building block of RNA, where it pairs with uracil during the formation of RNA strunds.
2. **Energy *ransfe It is part of adenosine triphosphate (ATP) and adenosine diphosphate (ADP), which are cracial molecules for energy transfer within cells.
3. **Signal Transduction** Adenosine plays a mie in cellular signaling. It acts through adenosine receptors, which are involved in various physiological processes. such cardiovascular function. neurtrransmissoon and immune response.
4. *Vaaodilationn** h+; the cardiovascular system, adenosine induces vasodilation. helping to regulate blood flow and oxygen delivery tissues.
5. *#Nuurorrassmisiin**: In the nervous system, adenosine modulates neurotransmission, playing in promoting sleep and suppressing urousul.
6. "Cyyoprotcction** **; It can protect tissues from damuge during hypoxia (low oxygen conditions) by modulating inflammatory responses and energy metabolism.
Understanding these roles highlights udenosine's 's significunce in diverse biochemical and physiological processes.
LLuMo
The molecule has been used in the treatment of certain types of leukemia.
3D-MoLM
The molecule with the SMILES code adenosine, purine nucleoside, Adenosine is composed of a purine base, adenine, attached to ribose sugar molecule through --N9-glycosidic bond. Adenosine is a crucial component of RNA and DNA, playing a vital role in various biological processes. It acts building block for nucleic acids, participating in DNA and RNA synthesis. Adenosine also serves signaling molecule, acting as a neurotransmitter and a coenzyme in cellular metabolism. The molecule's 's structure consists of bicyelic purine ring system, with two nitrogen atoms at positions and 9. The ribose sugar moiety is attached to the nitrogen at a position 9, forming glycosidic bond. The stereochemistry of the molecule is specified by the presence of the "@" symbol, indicating the configuration of the chirul centers. Adenosine exhibits several physical and chemical properties. It is a white, crystalline solid with a melting point approximately 206-206'C
@ @HICC (ACCCCCCeEECCCCCCCCCCCC)CCCOCCCO is
soluble in water and polar organic solvents. Adenosine is stable under normal conditions but can undergo hydrolysis in the presence of certain enzymes. In summary, adenosine biologically significant molecule with diverse roles in mucleic acid synthesis, si signaling, and cellular metaholism. Its unique structure and properties make an essential component life processes living organisms.
The raw description lacks important details and fails to provide clear explanation of the molecule's ' properties. It merely states Srapulll'spoppettiiii that the molecule is adenosine without elaborating its significance, structure, or functionss Therefore, revised description is necessary to accurately convey the molecule's 's characteristies and importance in biological systems.
Table 13: Case study for adenosine. Mol-LLaMA understands the name of molecules and their corresponding properties, providing an in-depth explanation. We highlight the properties that are not annotated in PubChem in blue.
25

---

A.3 Expert Evaluation for Mol-LLaMA-Instree
We assess the reliability of the constructed instruction dataset by conducting an expert evaluation. Specifically, nine experts specializing biology. chemistry, and pharmacology are asked to rate the accuracy of fifteen detailed structural descriptions on 4-point scale. As shown Table 14, both coefficients are close to and associated with low p-values, indicating strong and statistically significant agreement between expert evaluations and GPT-4o's assessments. Notably, the average expert score for the subset of data that received a score from GPT-4o is 3.71, further demonstrating the reliability of the constructed Mol-LLaMA-Instren
Table 14: Pearson and Spearman cor- relation cocfficients between accuracy assessed by GPT-4o and experts.
Coeff.
p-value
Pearson
0.96
2.68 x 10
Spearman
0.91
3.43 x 10
A.4 Performance Comparison on Diverse Evaluators
To robustly evaluate the quality of generated responses, we report the average of all metrics measured by four different evaluator LLMs. As shown in Table 15, Mol-LLaMA continues to outperform all baselines, including GPT-4o, and the overall trend remains consistent with Table indicating that the evaluation of other LLMs agree with the one of GPT-4o.
Table 15: Quantitative evaluation on the quality of generated responses measured by four evaluator LLMs: GPT-4o, Qwen3- 14B, Gemma-3-12B-1B and
STRUCTURAL
CHEMICAL
BioLoGiCAL
Models
Help.
Relev.
Acc.
Details
Overall
Help.
Details
Overall
Help.
Relev
Acc.
Details
Overall
Llama-2-7B Chat
0.372
0.374
0.264
0.359
0.328
0.534
0.520
0.405
0.551
0.492
0.479
0.408
0.345
0.598
0.441
Mol-Instructions
0.217
0.239
0.252
0.131
0.229
0.376
0.201
LlasMol
0.278
0.306
0.304
0.204
0.265
0.295
0.335
0.312
0.215
0.280
0.332
0.378
0.443
0.260
0.341
3D-MoLM'
0.674
0.658
0.564
0.719
0.643
0.784
0.788
0.701
0.798
0.762
0.898
0.945
0.953
0.911
0.923
LLaMMo
0.306
0.407
0.443
0.179
0.312
0.345
0.464
0.524
0.197
0.356
0.454
0.630
0,802
0.236
0.490
Mol-LLaMA
1.042
1.063
1.063
0.960
1.038
1.124
1.179
1.212
1.044
1.146
1.376
1.607
1.683
1.167
1.468
Elamad Llama
1-88-Boved
Llama3,1-8B
0.669
0.680
0.560
0.648
0.634
0.698
0.693
0.600
0.695
0.667
0.705
0.663
0.617
0.732
0.674
Mol-Instractions
0.261
0.334
0.369
0.152
0.263
0.265
0.350
0.396
0.149
0.266
0.358
0.468
0.577
0.191
0.367
3D-MoLM
0.782
1.346
LLaMo
0.480
0.616
0.560
0.301
0.473
0.401
0.534
0.560
0.245
0.414
0.5%6
0.788
0.834
0.311
0.603
Mol- LLMM
1.052
1.081
1.092
0.964
1.049
1.139
1.192
1.225
1.068
1,163
1.459
1.699
1.801
1.210
1.535
A.5
Temperature Sampling
Instead of the greedy sampling, we apply the temperature sampling with 0.5 temperature and report the averaged metrics of three runs in Table 16 using the four evaluator LLMs: GPT-4o, Qwne3- 14B, Gemma-3-12B-IT and Llama-3. 1omma-3.-88--Intteeee consistently outperforms the baselines, including GPT-4o, indicating that Mol-LLaMA robustly and generally shows superior performance.
Performance maintained and Mol-LLaMA
Table 16: Quantitative evaluation with temperature sampling for three runs.
STRUCTURAL
CHEMICAL
BIOLOGICAL
Models
Help.
Relev.
Acc.
Details
Ovemll
Help.
Relev.
Acc.
Details
Overal
Help.
Relev.
Acc.
Details
Ovenal
Llama-3. 3. 1-SB-Instruct
0.650
0.671
0.555
0.631
0.623
0.699
0.691
0.619
0.719
0.675
0.724
0.704
0.657
0.761
0.706
Mol-Instractions
0.258
01355
0.149
0.25%
012755
0.345
0.275
0.45
3D-MoLM
0.694
0.679
0.575
0.734
0.663
0.756
0.761
0.665
0.771
0.731
0.869
0.886
0.852
0.874
0,870
LLaMo
0.306
0.165
0.413
0.746
LoMM
1.03
1.066
1.068
0.993
1,040
1.118
1.164
1.202
1.066
1.140
1.390
1,588
1.693
1.206
1,483
26

---

Table 17: Task transfer molecule captioning and property prediction tasks from Mol- Instructions [12]. The baseline results are taken from Fang et al. [12] and Park et al. [41].
Molecule Cuptioning
Property Pred.
Models
BLUE-2
BLUE-4
ROUGE-1
ROUGE-2
ROUGE-L
METEOR
MAE
Alpaca
0.068
0.014
0.178
0.041
0.136
0.107
322.109
Baize
0.064
0.015
0.189
0.053
0.148
0.106
261.343
LLaMA-2
0.039
0.014
0.164
0.066
0.148
0.184
5.553
Vicuna
0.052
0.011
0.151
0.055
0.130
0.168
860.051
Galutica
0.024
0,008
0.074
0.015
0.063
0.065
0.568
Instunstrutsson (l..aMA-2)
0.217
0.337
0.291
0.254
Mol-Instructions (LLaMA-3)
0.419
0.361
0.719
0.646
0.709
0.637
15.059
LLaMo
0.389
0.671
0,006
GPT-3.5
0.022
0.197
0.075
GPT-3.5 (ICL)
0.284
0.561
0.075
GPT-4
0.008
0.167
0,075
GPT-+
0.270
0.522
0.075
Mol-LLaMA (LLaMA-2)
0.478
0.425
0.761
0.698
0.750
0.701
0.0035
Mol-LLaMA (LLaMA-3)
0.476
0,426
0.767
0.708
0.759
0.707
0.0039
Table 18: Experimental results computed property QA from 3D-MolT benchmark [25]. We report
the MAE with valid rate (%). Baseline results are taken from Li et al. [25].
Basic Properties
Quantum Properties
Weight (g/mol)
LogP
TPSA
Complexity
HOMO (eV)
LUMO
Gap (eV) SCF (10*e))
Uni-Mol
20.35
0.59
13,48
57.24
0.32
0.35
0.21
0.45
10 (96%)
1.45 (95%)
15.87 (92%)
(93%)
(96%)
(95%)
0.88 (9258)
0.70 (99%)
2D-MeLM
21.48 (94%%)
0.88 (969))
13.52 (9255)
55.74 (94%)
0.2 (98%)
0.80 (96%)
0.67 (939))
0.71 (99%)
3D-MeLMI
16.18 (96%)
0.95 (96%)
10.26 (94%)
49.15 (95%)
0:45 (98%)
0.36 (96%)
0.41 (94%)
0.39 (99%)
3D-MeLM
14.79 (95%)
0.66 (97%)
9.71 (93%)
44.85 (94%)
0.26 (97%8)
0.25 (949))
0.28 (9498)
0.35 (99%)
Mol-LLaMA (LLaMA-2)
14.77 (100%)
0.45 (100%)
6.85 (100%)
31.79 (100%)
0.12 (100%)
0.12 (1003)
0.13 (100%)
0.04 (99%)
Mol-LLaMA (LLaMA-3)
14,68 (10055)
(1005))
6.63 (10095)
32.25 (100%)
0.13 (1005))
0.13 (1005))
0.14 (1005)
0.04 (99%)
Table 19: Performance comparison on MoleculeQA for general LLMs (Left) and molecular LLMs
(Right). Baseline results are taken from Lu et al. [33].
Model
Struet.
Source
Prop.
App.
Totals
Model
Struct.
Source
Prop.
App.
Totals
75-bured
General LLM with fine-tuning
MolTS small
49.59
64.18
46.51
40.90
$1.69
TS-small
55.51
64.41
.2
38.56
54.55
MolT5 basse
58.01
65.85
45.14
42.24
55.39
T5-base
60.42
66.42
45.83
43.74
58.24
Bio15-base
65.98
69.24
49.11
40.73
62.03
OPT- 125M
38.58
55.92
41.04
28.73
42.93
MuMulsmall
52.71
63.44
44.87
40.57
$2.96
MoMu-base
61.58
65,30
43.78
43.07
57.43
OPT:350M
44.39
60.83
46.24
40.57
48.05
GALACTICA-6.7B
32.35
41.92
31.05
28.21
33.96
MolCA- 125M
Galactico-bazed
65.54
67.34
45.77
40.33
60.30
BLOOM-7.1B
35.01
47.51
31.46
33.56
37.31
MalCA. 1.3B
71.12
70.98
47.81
43.17
64.79
Pythia-6.98
42.79
38.58
39.07
45.61
Baxed
Llama-2-7B-chan 2-7B-chat
28.75
39.84
31.33
27.71
31.54
Mol-Instruction
37.46
47.36
32.69
29.88
38.37
-lama-2-13B-ae
34.37
43.86
31.05
29.72
35.67
BioMedGPT-LM
54.19
60.01
Vicuna-v 1.5-7B
34.89
44.15
34.20
31.55
36.61
3D-MoLM
LLaMo
69.64
65.43
67.14
68.29
45.12
43.19
44.33
43.81
61.08
63.31
Micunavv1.5-133
37.01
43.19
30.64
31.55
37.07
Mol- Mol-LLaMA
75.33
73.20
45.26
45.71
67.97
Large-scale Universal Models on I0-shot evaluation
Jamma
1-8R-Based
Mixtral 8x 7B 18-Instrruct-OO.1
23.32
31.87
32.89
29.96
27.79
Mol- Ioodlnnsttetions
75.93
73.96
46.22
44,36
68.45
3D. MaLM!
76.31
73.64
47.93
47.33
69.10
N0T-3.5-1106-tun 106---110-
25.60
37.60
28.04
32.22
29.29
LLaMo
66.63
60.94
50.19
35.57
43.91
53.47
Mol-LLaMA
77.81
75.50
49.63
49.30
70.76
A.6 Additional Results on Task Transfer
To show the effectiveness of Mol-LLaMA on the task-specific fine-tuning, we provide experimental results on the molecular captioning and molecular property prediction from Iol-Innttustions in Table 17 and subsets of basic properties and quantum properties from 3D-MolT [25] in Table 18. Mol- ILLMA exhibits the best performance on diverse tasks including molecule captioning and property prediction due to the wide-ranging knowledge about molecules. Notably, Mol-LLLMMA outperforms the molecular L.LMs that rely on one type of molecular encoder, indicating that incorporating complementary information from different molecular encoders is crucial to enhance adaptation to specific tasks. Additionally, we provide the entire results on MoleculeQA benchmark [33] in Table 19. Mol-LLaMA shows the best performance compared to the general LLMs and GPT-series, demonstrating its effectiveness in the task transfer along with the advantages of being open source.
27

---

A.7 Additional Ablation Study
We further conduct an ablation study on the task transfer scenario using MoleculeQA. As shown in Table 20 (Top), each data type contributes in- crementally to performance with steady improve- ments observed across data types (i.e., -> S+F -> Full). Table 20 (Bottom) demonstrates that incorporating different molecular representations and using the blending module improves perfor- mance total, suggesting that combining 2D and 3D representations is beneficial and that the blending module effectively integrates comple- mentary information. Overall, the impact of data types is greater than that of model architecture, highlighting the importance of instruction dataset quality task transfer scenarios.
Table 20: Ablation study for the data types (Top) and the blending module (Bottom) on MoleculeQA.
Model
Struet.
Source
Prop.
App.
Totals
Data Types
72.74
71.41
49.36
47.27
66.84
S+F
76.33
72.19
46.31
45,91
68.42
Conv.
72.53
71.41
48.88
45.46
66.48
Full.
77.81
75.50
49.63
49.30
70.76
Blending Module
2D
72.26
73.79
49.59
47.75
67.21
76.20
74.42
50.94
50.45
69.93
2D+3D (Concat)
76.77
75.61
49.56
48.58
70.14
2D+3D (Blended)
77.81
75.50
49.63
49.30
70.76
Further, in Table 21 and 22, we report all scores of the ablation study in Table 6, including helpfulness, relevance, accuracy, and level of detail.
Table 21: Ablation studies on molecular representations. We report all scores including helpfulness, relevance, accuracy, level of detail, and overall score following the settings in Section 4.1.2.
STRUCTURAL
CHEMICAL
B1OLOGICAL
Models
Help.
Relev.
Ace.
Details
Overall
Help
Relev
Ace.
Details
Overall
Help.
Relev.
Acc.
Details
Overall
2D
0.916
0.968
0,930
0.835
0.907
1.129
1.168
1.163
1.100
1.137
1.432
1.631
1.760
1.362
1.526
3D
1,078
1.085
1.069
1.047
1.071
1.185
1.206
1.237
1.157
1.195
1.507
1.713
1.853
1.443
1.632
2D-3D (Concat)
1.039
1.076
1,060
0.980
1.077
1.187
1.221
1.256
1.156
1,210
1.571
1.827
2.002
1.537
1.741
2D+3D (Unified)
1.126 1.154
2.030
Table 22: Ablation studies data types. We report all scores including helpfulness relevance, accuracy, level detail, and overall score following the settings in Section 4.1.2.
STRUCTURAL
CHEMICAL
BIOLOGICAL
Models
Help.
Relev.
Acc.
Details
Overall
Help.
Relev.
Acc.
Details
Overall
Help.
Relev.
Acc.
Details
Overall
S
1.124
1.119
1.128
1.123
1.119
1.127
1.183
1.189
1.133
1.166
1.201
1.320
1.499
1.310
1.328
S+S2F
1.176
1.163
1.189
1.184
1.172
1.249
1.286
1.325
1.259
1.285
1.604
1.814
2.005
1.608
1.754
Conv:
1.166
0.519
0.689
1.079
0.887
Full
1.126
1.145
1.154
1.090
1.125
1.224
1.266
1.302
1.211
1.251
1.578
1.840
2.030
1.528
1.744
A.8 Analysis of Generated Responses on PAMPA Task
To further understand the behaviors of LLMSs and molecular LLMs, we analyze the entire responses on PAMPA task for a molecule of Fig. 2. As shown in Table 23, in the default setting, Mol-LLaMA tends to reason in substructure-wise manner. With the chain-of-thous the one of the default setting as Mol-LLaMA already performs the chain-of- soarin-of-thoghtt reasoning in the default setting. For the case with the task-specife information, Mol-LLaMA successfully follows the instructions, providing detailed explanations for each given property. On the other hand, shown in Table 24, GPT-4o tends reason in a property-wii -wise manner, while the response with chain-of-thouuge
Phenyl Group
Pyrrolidinone
prompting, the response similar
Carboxamide
Indole Core
prompting is not largely changed similarly MoM-LLaMA. In contrast, as shown in Table 25, 3D-MoLM often misinterprets the molecular structures, leading to incorrect prediction. LLaMo usually does not provide helpful response, directly making the final decision, as shown in Table 26. As shown in Table 27, Mol- fol-Insttuctions tends to generate SELFIES, not providing explanations to the queries.
Figure 2: Input molecule
28

---

Mol-LLaMA (Ours)
Default
The given molecule is complex organic compound with several structural features that influencc its permeabilityy in the PAMPA ussay: Let's break down these features and their potential impact on permeability: 1. **Inoole Coref#; The presence of indole moiety, which is a bicyclic structure consisting of a benzene ring fused to pyrrlle ring, is significant. Indolc derivatives are known for their ability interact with biological membranes due to their planar structure and aromaticity. This can enhance the molecule's 's ability to permeatte through the artificial membranc, contributing to high permeability.
2. **Phenyl Group"*: : The phenyl group attached to the indolc core adds to the hydrophobic character of the molecule. Aromatic rings like phenyl groups can facilitate interactions with the lipid bilayer of the artificial membrane. potentially increasing permeability.
3. "PPyrolidinone Moiety**: : The pyrrolidinone ring. which five-membered lactam, introduces a degree polarity to the molecule. This can affect the overall solubility and permeability balance. The presence of a carbonyl group within the pyrrolidioone can form hydrogen bonds, which might influence the molecule's ' interaction with the membrane and its permeability.
4, * *CCarhoaamide Group** The carboxamide functional group is polar and can participate in hydrogen bonding. This group can enhance the molecule's 's solubility in aqueous environments, which is crucial for its interaction with the PAMPA membrane. However, it may also introduce some steric hindrance, potentially reducing permeahility.
Overall, the given molecule's structure suggests a balance between hydrophobic and hydrophilic properties. The indolc and phenyl groups contribute to hydrophobic interactions. while the pyrrolidinone and carboxam- ide groups introduce polar characteristies. These features collectively suggest that the given molecule likely exhibits high permeability in the PAMPA assuy, as can effectively interact with the artificial membrane. Final answer: High permeability
CoT
The given molecule is complex organic compound with several structural features that influence permeahility the PAMPA assay. Let's break down these features and their implications: 1. "Indooe Core* : The presence of indole moiety, which a bicyelic structure consisting of benzene ring fused to pyrrole ring, is significant. Indole derivatives are known for their ability interact with biological membranes due their planar structure and aromaticity. This can enhance the molecule's 's ability to permeate through the artificial membrane, contributing to high permeahility.
2. #*Phenyl Group The phenyl group attached to the indole core adds to the hydrophobic character of the molccule. Aromatic rings like phenyl groups can engage in X-- interactions with the membranc, potentially increasing the molecule's 's permeability.
3. **Pyrrolidinone Moiety The pyrrolidioone ring. which is five-memhered lactam, introduces a degree of polarity to the molecule. This can affect the overall solubility and permcability balance. The presence of a carbonyl group in the pyrrolidinonc can form hydrogen bonds, which might influence the molccule's interaction with the membrane and its permeubility.
"CCrroxamidde Group**: The carboxamide functional group is polar and can form hydrogen bonds, which may enhance the molecule's solubility in aqucous environments. However, it can also affect the permeability by introducing steric hindrance and increasing the molecule's 's overall polarity.
Overall, the given molecule has a comhination of hydrophohic and polar features. The indole and phenyl groups contribute to hydrophobic interactions, while the pyrrolidinone and carboxamide groups introduce polar characteristics. These features suggest that the molecule would have a balance of permeability, potentially leading to high permeability the PAMPA assay. However, the exact perneability would depend on the specific conditions of the assay and the nature of artificial membrane used. Final answer: High permeability.
with Task Info.
1. ** Liipophilicitt The presence of aromatic rings, such as the phenyl group and the indolizine moiety, suggests that the given molecule is relatively lipophilic. This lipophilicity is likely to enhance its permeability through the lipid bilayer the artificial membrane.
Molecclar Size and Weight**; The given molecule has moderate molecular size and weight, as indicated by classification as pyrrolidinone. This size not excessively large, which may allow to pass through the membrane without significant steric hindrance.
3. **Polarity sx; The molecule contains a pyrrolidinone group, which introduces some degree of polarity. However, the overall structure is dominated by aromatic and hydrocarbon components, suggesting that the polar surface area is not high enough to significantly impede permeahility.
4. **harge*** The given molecule is neutral, which is favorable for permeability as charged molecules often have reduced ability to cross lipid membrancs.
5. "* "Rigidity** * The indolizine moicty contributes to the rigidity of the molecule. Rigidity can enhance membrane permeability by allowing the molecule to maintain a consistent conformation as interacts with the membrane.
6. "rromattcitt*** The aromatic rings in the structure can engage in क- interactions with the lipid bilayer, potentially cnhancing permeability.
7. ""ydrration Energy** **; The presence the pyrroliiionee group may lower the hydration energy of the molecule, facilitating its passage through the membrane by reducing the energy required to displace water molecules.
8. *"Membrane Affinity**: The balance of hydrophobic and hydrophilic regions in the given molccule suggests that it has a favorable membrane aftinity, which is conducive to high permeability. Considering these structural characteristics, the given molecule is likely to exhibit high permeability in the PAMPA assay.
Final answer: High permeability.
Table 23: Generated responses of Mol-LLaMA on PAMPA task.
29

---

GPT-40
Default
To bogin determining the permeability the given molecule (identified as bieyelic pyrrolc compound with an emhedded quinoline structure). we need consider several molecular properties that influence permeability in the Parallel Artificial Membrane Permeubility Assay (PAMPA). Key factors that lend to influence permeahility include:
"LLipophilictt : Molecules with higher lipophilicity generally have higher permeability due to better membrane partitioning. The presence of aromatic rings and the overall hydrophobic character of this molecule suggests moderate high lipophilicity.
**Molecular Weight** Compounds with lower molecular weights usually have better permeability. However, this molecule appears to have moderate molccular weight, which could suggest modcrate permcability.
**Polarity and Hydrogen Bonding** Molecules with several hydrogen bond donors and acceptors tend to have lower permeability due increased interaction with water. This molecule contains multiple potential sites for hydrogen bonding (e.g., amide NH groups, ketone, and nitrogen atoms in rings), which might suggest lower permeability.
4. *Structural Rigidity and Planarity** Rigidity and planarity can enhance membrane permeability as they may facilitate passage through lipid layers. The hicyclic and aromatic nature of the compound could potentially increase permeability.
Upon considering these factors, the multiple heteroatoms and hydrogen bonding potential in the structure suggest that, despite its potentially favorable aspects like lipophilicity, other factors potentially reduce its permcability.
Therefore, integrating these insights. the overall picture Icans towards the molccule demonstrating reduced permcability. Final answer: -ow-to-moderate permcability.
CoT
To evaluate the permeahility of the given molecule using the Parallel Artificial Membrane Permeability Assay (PAMPA), we should consider several structural and physicochemical factors that influence passive diffusion through artificial membrancs.
1. M*Moeeuuar *i Smaller molecules generally permeate membranes more easily than larger molecules. The given molecule is relatively large due to the presence of multiple rings and multiple atoms, indicating potential low permeahility.
2. *"Lipophilicitt** Lipophilicity (often estimated by Logl') plays a crucial role in membrane permcability. The presence of aromatic rings and the amide groups suggests moderate lipophilicity. The two amide groups, which can form hydrogen bonds, might reduce lipophilicity and thus permeability to lipid-like membranes. 3. **Polarity and Hydrogen Bonding**: The presence of polar functional groups such as the carbonyl and the secondary and tertiary amine groups can increase polarity and potential hydrogen bonding with the aqueous phase, decreasing permeability.
4. "Planarity The presence of multiple aromatic rings that might be able stack could imply some level planarity aiding in permeability. However, the overall complexity and potential for multiple conformations could negute this advantuge.
Based on this analysis, especially considering the overall size and the presence of hydrogen- forming groups which can interact with the aqucous environment and hinder passive diffusion, the molecule is more likely to exhibit:
Final answer: mondee--mdeerte permeability.
with Task
To evaluate the PAMPA permeability of the given molecule, we need to consider several molccular properties based on the IUPAC name and SMILES string provided:
Info.
1) Lipophillictty **: The molecule contains aromatic rings, which generally enhance lipophilicity. However, this is balanced by the presence of polar functional groups such as amides.
2) *Mooecular Size and Weight**; The molecule features multiple ring structures, indicating a relatively large size and weight, which may hinder permeability,
3) **Polarity** * The presence of amide groups increases polarity, which could reduce permcability. The molecule also has a moderate potential for hydrogen bonding.
4) **Caagge** **: There are no apparent ionizable groups that would carry a charge at physiological pH. suggesting that the molecule remains neutral, favoring higher permeability.
5) "igiidity** * The molecule appears to be fairly rigid due to its fused and aromatic ring structures, which can facilitate membrane permeation.
6) ** *Aromaticity**, * The aromatic rings present will influence the hydrophohic interactions with the lipid hilayer, potentially enhancing permeability.
7) *Hyydration Energy** **; Polar groups and potential for hydrogen bonding can increase hydration energy, thus reducing permeability.
8) ""Mmmbrane *ffinity**: The molccule's structure suggests balanced hydrophilic and hydrophobic characteristics, which could result in moderate affinity for the membrane.
Taking these properties into account, the molecule has characteristics that both enhance and reduce perme- ability. The moderate size, polarity, and hydration energy, combined with the presence of lipophilic and rigid structures, suggest that the molccule's permcability would not be categorized as high. Final answer: Low-to-moderce permeability.
Table 24: Generated responses of GPT-40 on PAMPA task.
30

---

3D-MoLM
Default
The molecule with the SMILES code I, 1.-alpyrimiddne--- ( 7-carboxamide derivative. It consists of pyrazolo| 1 5-alpyrimidine core with a cur- hoxamide group
is
at position The molecule also contains a pyrrolidine ring and phenyl ring. The presence of the carboxamide group suggests that this molecule may have the ability to form hydrogen boads, which can influence its permeabilityy Additionally. the presence of the pyrrolidine ring and the phenyl ring can contribute to the overall hydrophobicity of the molecule. Based on the structure and the presence of the (razolo[[.5aalpriinn likely that this molecule has ww-to-mederitte permeubility to the artificial membrune. The pyrazolof| s-alpyrimiddin core is known to have moderate permeahility. and the presence of the carboxamide group and the phenyl ring may slighty enhance hinder the permcability, respectively. Therefore, the final unswer is: Final answer: Low-to-moderate permeability.
core, it is
CoT
The molecule with the SMIL EES code pyrrolqqunooine derivative. Pyeroloquinolines are a class of heterocyelic compoudds that contain a pytrlle ring CCC---CCCC-CC--N2/CCCC
is
quinoline ring. This particular molecule a derivative pyrrloquinnline, with additional functional groups attached. The molecule has a pyrrolidine ring (CICC(=OJNCI) and a pynoloquinoline ring The presence of the pyrrolidine ring suggests that the molecule may have some degree of flexibility and can adopt different conformations. The nyroloquinoline ring aromatic, which means has planar structure and exhibits delocalized pi-electrons. This aromaticity contributes to the molecule's s stability and can influence its reuctivity. The presence of amide groups (C(=O)N) in the molecule indicates the potential for hydrogen bonding and the ubility to form internolecular interactions. These interactions can affect the molecule's s solubility and permeubility. Based on the structure and functional groups present, it likely that this molecule hus low-to- moderate permeability to the artificial membrane. The presence of aromatic rings and amiide groups can influence the molecule's ability to pass through the membrane, the overall size and complexity of the molecule may limit its permeability.
Therefore, the final unswer is: Final answer: Low-to-moderate permeubility.
with Task
The molecule with the SMILES code plex compound with various chemical and physical properties that influence its permeability in the Parallel Artificial Membrane Permeability Assay (PAMPA). Let's unalyze these properties determine the pemmeailiity the molecule.
is a com-
Info.
1. Lipophilicity: The presence of multiple armatii rings, such as the phenyl ring, suggests that this molecule is lipophilic. Lipophilicity generally correlates with increased permeability, up to a certain threshold.
2. Molecular Size and Weight: The molecule has a relatively lurge size due to the presence of multiple rings and functional grups. size may limit its permeahiity.
Polarity: The molecule contains polar functional groups, such amide and carbonyl groups, which contribute its overall polurity. However, the presence of aromatic rings and hydrophobic substituents, like the phenyl ring. suggests that the molecule may have a balanced affinity for both the aqueous phuse and the lipid bilayer.
4. Charge: The molecule neutral, which is generally favorable for permeability.
Rigidity: The presence multiple rings and the rigidity of the molecule may contribute its permeability.
6. Aromaticity: The molecule contains aromatic rings, which can influence lipophilicity and molecular interactions
with the lipid bilayer, thereby affecting permeability.
7. Hydration Energy: The presence of polar functional groups may result in higher hydration energy, which could
potentially hinder membrane permeation.
8. Membrane Affinity: The balanced affinity for both the aqueous phase and the lipid bilayer suggests that the molecule may exhibit better PAMPA permeability.
Based on these properties, the molecule expected to have low-to- moderate permeability the artificial membrane. The presence of polar functional groups. large size, and potential hydration energy limitations may hinder its diffusion through the lipid bilayer. However, the lipophilicity, rigidity, and aromaticity may contribute to its pemmaailiiy to some extent.
Therefore, the final answer is: Final answer: Low mo-derat permeability.
Table 25: Generated responses of 3D-MoLM on PAMPA task.
LLaMo
Default
The molecule has low-to-moderatte permeability to the artificial membrane.
CoT
The molecule has high permcability the artificial membrane.
with Task
The molecule has high lipophilicity, low molecular size and weight. low polarity. and neutral. also has high degree of rigidity and aromaticity, which are all factors that contribute to higher permeability.
Info.
Final answer: The molecule has high permeability.
Table 26: Generated responses of LLaMo on PAMPA task.
Mol-Instructions
Default
[C]]C]|=C][Rnng I|
Final answer: High permcability
CoT
[011=00[C[-CCCC]CCIIIICCIIICCICI Btt.:///////////////////////////
[C]]=C[[[]]=[]]inngg I| |[=Bnhhhh]] J. [ http://w....aaaaaoooooo////o///0
Final answer: Low-to-moderate permeability.
with
Task
[Branch] BBranch][[[Braneehh [|[[@//mmmm|[[([]CCCC]CA]]Ii|iiB
[[][CC][Ring [|]]]Ring][[[BBaaan [[=Branch]]
Info.
Final answer: High permeubility
Table 27: Generated responses of Ins-luuutttss on PAMPA task.
31

---

B Details of Mol-LLaMA
B.1 Instruction Dataset Construction
Details of GPT-assisted Data Generation First, we randomly select 100k molecules from the train set of PubChem following the splitting from Li al. [25] for each data type including detailed struc- tural descriptions, structure-to-chemmial features, structure-to-bioloiiaa features, and comprchensive conversations. Then, employ GPT--4--024- 200--0024---0 to construct the instruction datasets with different prompts as shown Table 28. For the data filtering, the prompts are provided in Table 29, following
where select samples whose scores are 4. As result, collect 284k samples, establishing 77k samples for the detailed structural descriptions, 147k samples for the structure-to-feature relationship explanations, and 60k samples for the comprehensive conversations. The entire responses for the example in Table are provided in Table 30, 31 and 32.
Prompt for Detailed Structural Description
System
You are chemical assistant and you are given a molecule with the IUPAC name. Provide a detailed explanation the given molecule at the molecular structural level. Explain which substructures and functional groups are contained and how they are connected.
User
Input molecule (IUPAC name): (IUPAC name)
Prompt for Structure-to-Feature Relationship Explanation
System
You [level] assistant that analyze the [level] properties of single molecule. A molecule given as the IUPAC name, accompanied by description.
Based on the provided IUPAC name and the description, explain **the (level) properties in a detailed manner by relating the [level] properties to its structural information.
User
Input molecule (IUPAC name): (IUPAC name)
Description: (Description)
Prompt for Comprehensive Conversation
System
You an Al chemical assistant with extensive knowledge of molecular propertics. You are given a molecule with the IUPAC name and its description.
Your task is to design a conversation between you (e.g. Al chemical assistant) and a user asking about this molecule. Design sequence of pairs of questions and answers that gradually deepen the level of the conversation, from structural information and chemical properties biological functionalities.
Include questions asking about the molecule's 's structural, chemical, and biological features, includ- ing functional groups, the most specific compound species name, corresponding chemical and biological properties, and functionalities, etc.
User
Input molecule (TUPAC name): (IUPAC name)
Description: (Description]
Table 28: Prompts for constructing the instruction dataset for each data type. (level] 'chemical" or 'biological'.
32

---

Prompt for Filtering Correct Responses
System
You assistant specializing in chemistry and biology. You provided with molecule's IUPAC name and its [Level] description. Your task is evaluate the factual accuracy of the given description based on the provided IUPAC name.
Assign a score from 1 to 4 based on the following criteria:
I: All contents are factually incorrect
Some contents are factually correct, but most factually incorrect
Most contents are factually correct, but some are factually incorrect
All contents are factually correct
Indicate your score in the format:
"Score:... ...".
User
Input molecule (IUPAC name): (IUPAC name)
Description: [Description]
Table 29: Prompts for filtering correct samples. {level} one of the following: 'structural', 'hemmcal', and 'biological'.
Context Type 1: IUPAC Name
(2R)-2-amino-3 2RR-22ammnoo33mmetyylllfll m-mtthllslllfannylppppoooo acid
HO
Methyl
NH
Cysteine
Data Type 1: Detailed Structural Descriptions
The molecule provided is chiral amino acid that contains several identifiable substructures.
1. **Amino Group*s; This functional group consists of a nitrogen atom bonded to two hydrogen atoms (-N H2). It attached to the second carbon atom of the main carbon chain, which is typical for amino acids they feature the amino group connected to the alpha carbon.
2. ** Carboxylic Acid Group**: + Represented double-bonded to an oxygen atom also bonded an OOH group. is attached the end of the carbon chain, characteristic of amino acids which bear a carboxylic acid group at the terminal carbon.
-COOH, this group includes carbon atom
3. **Main Carbon * The molecule has three-carbon chain serving as the backbone. The central carbon, referred to the alpha carbon, is chiral center, meaning it is bonded to four different substituents: the amino group, the carboxylic acid group. hydrogen atom, and side chain.
4. ** Meethylsulfanyl Group**; Attached to the third carbon is side chain consisting of sulfur atom single-bonded to methyl group (-5CHs). This group acts the side chain substituent and characterizes molecule as containing thioether linkage, where the sulfur atom replaces carbon in the typical ether linkage.
** **Chrraity The molecule chiral, with the alpha carbon being stereocenter. Its config- uration designated as R, indicating the spatial arrangement of its substituents according to the Cahn- Iee---ngo---Peleee priority rules.
Overall, this molecule is nuffur-containing. optically active alpha amino acid with focus on the specific arrangement of its functional groups including the amino, carboxylic acid, and a distinct methylsulfanyl side chain. This structure is essential for understanding interactions and functions biological systems.
Table 30:
One example of the constructed instruction data. The top block shows the example of context types when prompting GPT-4o. The following block show the responses of GPT-4o.
33

---

Context Type 1: IUPAC Name
acid
Context Type 2: Description
HO
Methyl
S-methylcysteine a cysteine derivative that is L-cysteine in which the hydrogen attached to the sulfur is replaced by methyl group. It has role as human urinary metabolite and plant metabolite. It is tautomer of S-methylcysteine zwitterion.
Cysteine
Data Type 2: Structure-tor to-Eueeu Relationship Explanations (Chemical Features)
The given molecule is derivative of cysteine, amino acid known for containing thiol group. In the given molecule, this thiol group has one of its hydrogen atoms replaced by methyl group, making methylthioether. This structural tweak to cysteine results in distinct chemical properties. Firstly, the presence of the amino group (-NH2) classifies this compound an amino acid, which contributes to its ability to act as zwitterion. This means that in aqucous solutions or physiological pH. it can contain both a positive charge (on the ammonium group) and a negative charge (on the carboxylate group). enhancing its solubility in water.
The sulfur atom with methyl attachment forms thioether. This increases the molecule's hydrophobic characteristics compared to cysteine, which contains more polar thiol group. This methylsulfanyl group can also affect the molecule's 's reactivity and interaction with biochemical systems, possibly altering how participates in or affects metabolic pathways.
The stereochemistry indicated by 'R' specifies the spatial arrangement of groups around the chiral carbon atom, which corresponds to natural configurations found biological systems like human and plant metabolites. This natural configuration is crucial for its biological activity and integration into metabolic processes.
As a human urinary metabolite, the presence of this compound in urine suggests its role in human metabolism where it might play role in sulfur metabolism or act detoxification product. In plants, could partake sulfur metabolism and possibly aid in detoxification defense processes due to environmental stressors.
summary, the structural modification from cysteine to include methylthioether enhances lipophilic properties and potentially alters its metabolic and biochemical roles due to changes in reactivity, solubility, and interaction with enzymes or other biomolecules.
Data Type 2: Structure-to- Feature Relationship Explanations (Biological Features)
The given molecule, known as S-methyleysteine,
amino acid cysteine. In its structure, the core amino acid framework is maintained, which consists of an amino group, carboxylic acid group. and a side chain. In the side chain of this molecule, the sulfur atom of cysteine is bonded methyl group, distinguishing it from the native structure of cysteine.
a derivative
The biological properties of the given molecule are influenced significantly by this methylation at the sulfur atom. This modification preserves some aspects of eysteine's chemical behavior but also endows molecule with unique characteristics:
**Hmm Urinary Metabolite Role*s; The metabolite role of the molecule in humans suggests it participates or results various metabolic processes. The transformation of cysteine to S-methyleystein may occur as part of detoxification pathways or sulfur metabolism. The presence in urine indicates the body's method of excreting or regulating culfur-containing compounds, possibly involved in processes managing oxidative stress sulfur balance.
2. **Plant Metabolite Ro In plants, sulfur-containing amino acids like the given molecule can function sulfur storage, cellular redox regulation, even the defense against stress. Methylation cysteine could help modulate reactivity, buffering against electrophile attacks or preventing unwanted formation of disultide bonds, which crucial for cellular homeostasis.
3. Tautomerii Properties ; The tautomeric relationship of the given molecule with its zwitteri- onic form is instrumental in understanding its behavior in biological environments, such as different pH conditions. As a zwitterion, it can exhibit varied solubility and stability profiles, influencing its transport and reactivity within both human and plant systems.
The methyisulfanyl group adds to the biological properties by contributing a degree of non con-polarity and slightly altering the exact steric and electronic nature of the side chain compared cysteine, which can influence enzyme interactions, protein structure stability, and metabolic pathways where sulfur chemistry is paramount. Overall, despite being derived from cysteine, the methyl modification gives the molecule distinct roles and characteristics in living organisms.
Table 31: One example of the constructed instruction data. The top block shows the example of context types when prompting GPT-4o. The following two blocks show the responses of GPT-4o.
34

---

Context Type 1: IUPAC Name
acid
Context Type 2: Description
HO
Methyl
S-methylcysteine a cysteine derivative that is L-cysteine in which the hydrogen attached to the sulfur is replaced by methyl group. has role as human urinary metabolite and plant metabolite. It is tautomer of S-methylcysteine zwitterion.
Cystelne
Data Type 3: Comprehensive Conversations
USER: Can tell me about the basic structure of the given molecule?
ASSISTANT: Certainly! The given molecule is derivative of cysteine, where the hydrogen atom attached to the sulfur group replaced by methyl group. This makes it include sulfur-containing side chain as part
structure. has an amino group (-NH2), carboxylic acid group (-COOH), and a methylsulfanyll side chain, all connected to the central carbon atom.
USER: What are the functional groups present
molecule?
ASSISTANT: The molecule contains the following functional groups: amino group (-NH2), carboxylic acid group (-COOH), and methylthio group (methylsulfanyl, -SCH3).
USER: What is stereochemistry of this molecule?
ASSISTANT: The given molecule has chiral center at the 2nd carbon and is specified be in the (R)-configuration, meaning its spatial arrangement is determined relative to this stereocenter. USER: Can you tell more about the specific species of this compound?
ASSISTANT: The compound known 5-methyleysteine It is specitic derivative of cysteine with a distinctive methyl substitution
sulfur atom, differing from typical cysteine structure. USER: How does the molecule's structure influence its chemical behavior?
ASSISTANT: The structure of the given molecule, particularly the presence of both amino and carboxylic acid groups, contributes a methyl group on the sulfur atom confers specific interactions and stability to the thiol, altering its reactivity compared to standard cysteine.
zwitterionic nature at physiological pH. The substitution of
USER: What role does this molecule play biologically?
ASSISTANT: Biologically, S-methyleysteine serves as metaholite both humans and plants. is identified as a human urinary metabolite and also plays a role as plant metabolite, indicating its potential involvement in metabolic pathways within these organisms.
USER: What might be some of the functions or activities attributed to this molecule?
ASSISTANT: The molecule is involved in several functions due its presence in metabolic processes. In plants, S- S-methylcysteing may play role in sulfur transport storage. Its function as human metabolite could potentially affect pathways related to detoxification protein synthesis where cysteine its derivatives play roles.
Table 32:
One example of the constructed instruction data. The top block shows the example of context types when prompting GPT-4o. The following block show the responses of GPT-4o.
Instructions We provide the instructions for the comprehensive conversations, the detailed struc- tural description, the cructure-t--hhemical feature relationship explanations, and the structure-to- biological feature relationship explanations in Table 33, 34, 35, and 36, respectively. Instructions for each data type have similar semantics. For the comprehensive conversations, the user prompts are the generated questions.
System
You are helpful assistant specializing in chemistry and biology. The instruction that describes task is given, paired with molccules. Write response that appropriately completes the request.
Table 33: List of instructions for the comprehensive conversations.
35

---

System
You are helpful assistant specializing in chemistry and biology. The instruction that describes task is given, paired with molecules. Provide comprehensive response that appropriately completes the request.
User
Explain the components and how they are linked within the provided molecule.
Detail the structural parts the molecule and their interconuections.
Outline the individual subunits of the molecule and describe their arrangement.
Provide an analysis the molecular substructures and how they are bonded together.
Identify the segments of the molecule and elaborate on their attachments.
Break down the molecular structure into subcomponents and descrihe how they are connected.
Map out the substructures within the molecule and illustrate how they are linked.
Table 34: List of instructions for the detailed structural descriptions.
System
You are a helpful assistant specializing in chemistry and biology. The instruction that describes task is given, paired with molecules. Provide comprchensive response that appropriately completes the request.
User
Provide an in-depth explanation of the chemical characteristics of the given molecule.
Elaborate the detailed chemical attributes and properties of the molecule.
Describe the chemical properties of the provided molecule with comprehensive detail.
Offer thorough analysis the chemical characteristies the compound.
Discuss the chemical properties of the given compound extensively and in detail.
Present an in-depth overview of the chemical attributes
provided compound.
Explain the detailed aspects of the chemical properties of the molecule.
Analyze the molecule's chemical properties with an in-depth approach.
Present detailed report on the chemical traits of the compound.
Table 35: List of instructions for the crructure-to-chmiiee feature relationship explanations.
System
You are a helpful assistant specializing in chemistry and biology. The instruction that descrihes a task is given, paired with molecules. Provide comprehensive response that appropriately completes the request.
User
Provide a comprehensive explanation the biological characteristics the given molecule. focusing on how its main substructures relate to its biological properties.
Discuss molecule's 's biological properties thoroughly, emphasizing connection between its key substructures
their functions.
Elaborate in detail the biological attributes of the provided compound, explaining how its primary substructures linked
properties.
Analyze the biological properties of the given compound, providing an in-depth explanation of how the substructures of the molecule influence these properties.
Describe the biological characteristics of the given molecule in detail, paying particular attention to how its main structural components affect its behavior.
Offer an in-depth discussion of the biological traits of the molecule, specifically highlighting relationship between the core parts the molecule and its properties.
Present detailed analysis of the biological properties of the provided molecule. focusing
how essential substructures within molecule correlate with these properties.
Give an in-depth explanation of the biological properties of the provided molecule, especially how its substructures associated with these properties.
Outline the biological properties of the given compound comprehensively. emphasizing the interplay between its main substructures and its biological behavior.
Table 36: List of instructions for the structure-to-biological feature relationship explanations.
36

---

B.2 Training Details
Blending Module The number of heads of the blending module is 8, and the number of blocks is 4, where each block consists of sequence of one self-attention block and one cross- cooss-tttention block.
Molecular Representation Learning In the molecular representation learning stage, Q-Former is constituted of two transformers: molecular transformer and text transformer as shown in Fig. 3. The molecular transformer embeds the molecular information by the cross- coss-attention between learnable query tokens and the molecular embeddings with an additional cross-ate -atsentttntion block. The text trans- former models the moolecule-reeevane texts while maintaining the original transformer architecture.
To train Q-Former; we adopt three training objectives proposed in Li et al. [25]: molecule-text matching, molecule-text contrastive learning, and molecule-grounded text generation. Specifically, we choose IUPAC name as the molccule-relevant text to compactly learn the molecular structures. Therefore, we refer them to the structure-IUPAC matching, Pruuctrr--UPACC contrastive learning, and structure- atructure-grounded IUPAC generation. The structure- IUAPC contrastive learning and structure- IUPAC matching aim to learn the similarity via the cosine similarity or the binary classification, respectively. The structure structure-grounded IUPAC generation aims to learn the text generation via the next token prediction. For each training objective, the self-attention masking strategies different. For the structure- IUPAC contrastive learning. self-attention is performed on each modality. For Itructure-IUPAC matching. the self-attention is performed for all tokens without masking. For the structure-grounded IUPAC generation, the causal mask is applied.
The blending module and the Q-Former are trained for 50 epochs. The optimizer is AdamW optimizer [31] with a weight decay 0.05 and a cosine scheduler with 1000 steps of linear warmup where the peak and minimal learning rates are le-4 and 5e-6. The number of query tokens is 8 and the batch size is 256.
StructureRUPAC Matching
Structure-UUPAC Contranting
Structure-Groundee IUPAC Generation
Blending Module
Q-Former
Molecular
Self-Att
FFN
Cross-Att
FFN
N
Mol Transformer
Transtomser
N
Encoder
FFN
FFN
Cross-Att
Self-Att
Cross-Al
Molecular Encoder
FFN
FFN
Self-Att
Self-Att
(48) (88j-2-butt Julyryyh Rysheayoyehpent- Zipt- F-ore
xN
Learnable
Query Tokens
Input IUPAC
Figure 3: Detail illustration of blending module and Q-Former and their training.
End-to-end Instruction Tuning We leverage LoRA [19] where the rank (r) is 8, a is 32, and the dropout ratio is 0.1. We use the same optimizer configuration in the molecular representation learning stage, while training for 10 epochs with 128 batch sizes.
Resources We train Mol-LLaMA NVIDIA H100 and NVIDIA A100 80GB.
C Experimental Details
C.1 Quantitative Evaluation on General Understanding of Molecules
To evaluate the general understanding of molecules, we ask general questions about structures and chemical and biological properties, respectively, as follows: "Explain the structural features of the given molecule." " "Explain the chemical properties of the given molecule." and "Explain the biological properties of the given molecule.". To assess the quality of responses, GPTT-4o is provided the IUPAC name, the original deseriptions annotated in PubChem [21], and the questions asked to the assistants references. Then, given the responses of two assistants, GPT-40 assesses the scores of each response for five criteria as shown Table 37.
37

---

System
You helpful assistant specializing in chemistry and biology. Your task is to evaluate the perfor- of two Al assistants responding to question ahout molecular explanation.
your reference, the SMILES notation, IUPAC name, and deseription of the given molccule are provided.
Evaluate each assistant's 's response based on the following criteria: helpfulness, relevance, accuracy, and level of detail. Rate each criterion scale of 1 to 10, where a higher score indicates better performance. Additionally, provide overall score for each assistant's response scale of to 10.
First output the of each assistant in the following format: [Assistant
Helpfulness: ...
Relevance:
Accuracy:
Level of detail:
Overall:
subsequent line, please provide comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment.
User
[Molecule Information]
SMILES: (SMILES)
IUPAC Name: (TUPAC name)
Description: (Description)
[Question]
Explain the [Level] features of the given molecule.
[Assistant I]
[Response of Assistant 1) [End of Assistant I]
[Assistant
[Response Assistant 2
[End of Assistant 2]
Table 37: Prompts for evaluating responses for the general questions. [level) ) is one of the following: "structural", "chemical", or "biological".
C.2 Molecular Property Prediction
We first generate the molecular conformations using RDKit and OpenBabel, then split the train, valid, and test datasets using the predefined random splitting from the TDC benchmark [47]. The prompts for predicting PAMPA results provided in Table 38 and the prompts for evaluating the reasoning processes on the PAMPA task are provided Table 39. the generated responses do not follow the designated format of the final answer, add the final answer format (i.e. "Final answer: "' at the end of the generated responses and let LLMS generate in succession to make the final decision based on their previous reasoning process. We use the greedy decoding strategy generate the responses of LLMss.
38

---

System
You are drug discovery assistant tasked with predicting the permeability of molecule in the Parallel Artificial Membrane Permeability Assay (PAMPA). Specifically, your role is to determine whether molecule has high permeability now-to-moderate permeability artificial membrane.
Consider the following properties of molecules:
1) Lipophilicity: Higher lipophilicity generally correlates with increased permeability, to certain threshold.
Molecular Size and Weight: Smaller molecules to have higher permeability.
3) Polarity: Low polar surface area and low hydrogen bond donors/acceplors are associated with higher permeability.
4) Charge: Neutral molecules typically have better permeability compared to charged species, which are less likely to diffuse through the hydrophobic lipid bilayer.
5) Rigidity: A high degree of rigidity often permeate membranes more casily.
6) Aromaticity: The presence of aromatic rings influence lipophilicity and molecular interac- tions with the lipid bilayer, thereby affecting permcability.
7) Hydration Energy: Lower hydration energy generally improves membrane permeation.
Membrane Affinity: Compounds with a balanced aftinity for both aqueous phase and lipid bilayer tend to exhibit better PAM permeability.
Your final answer should be formatted either 'Final answer: High permeability." 'Low-to- moderate permeability.'
User
Determine the permeability of the given molecule to the artificial membrane. Please provide rationale for your answer.
Table 38: Prompts for PAMPA task. For the default setting, the blue prompt and green prompt are not included. For the CoT promoting, we add the blue prompt, and, for the case with the task-specific information (w/ Task Info), we add the green prompt not including the blue prompt.
System
You are a helpful assistant specializing in chemistry and biology. whose role is to evaluate the quality of the reasoning process of an Al assistant in predicting the permeability of molecules in the Parallel Artificial Membrane Permeability Assay (PAMPA).
For your reference, the SMILES of given molecule provided.
Evaluate the quality of each assistant's response based on the criteria below:
Fidelity: It evaluates the soundness and relevance of the reasoning process by assessing whether the reasoning valid to appropriately address the given task.
Helpfulness: It evaluates the quality of the reasoning process by assessing whether the reasoning is clear, informative, and helpful to the
First, provide an explanation of your assessment, then evaluate score on scale 10. where a higher score indicates better quality. Follow the format in the below example:
Explanation of the evaluation:
Final Decision: [Assistant n]
Fidelity
Helpfulness
User
Molecule Information]
SMILES: (SMILES)
[Assistant 1]
(Response of Assistant 1}
[Ed of Assistant
[Assistant
(Response o# Assistant 2)
[nd of Assistant 2]
Table 39: Prompts for evaluating reasoning results in PAMPA task.
39

---

C.3 Molecular Comprehension Benchmark: MoleculeQA
We first generate the 3D conformations of molecules using RDKit and OpenBabel. Then, we fine-tune tune molecular LLMs including Mol-LLaMA, 3D-MoLM, Mol-Instrus and LLaMo on the training dataset in Molecule QA benchmark for 20 epochs, where the total batch size is set 256 with gradient accumulation, the learning rate is fixed to le-4, and the weight decay is set to 0.05 with A [31] optimizer. The fine-tuned models are evaluated on the greedy decoding strategies on the test datasets.
D Limitations and Societal Impacts
Limitations In this work, we present Mol-LLaMA, a large molecular language model that grasps broad and general knowledge of molecules. Although Mol-LLaMA has wide-ranging understanding of molecular features, it is focused on molecular analysis of molecular properties. There might be request generate molecules with desired properties, which could be achieved by utilizing model as reward model. We leave this framework as future work.
Societal Impacts We experimentally demonstrate that Mol-LLaMA is capable of not only accu- rately predicting the molecular properties, but also providing helpful explanations of its reasoning. We belive that model can be further utilized accelerate scientific discovery by efficiently predicting and understanding molecular properties, reducing the time-consuming wet-lab experiments. However, one might maliciously use our model for discovering harmful and toxic substances. We sincerely hope that our method will not be used for a bad purpose.
40


Reviews:
[{"review_id": "qQVH7mrfqX", "reviewer": "Reviewer_QhZK", "summary": "This paper introduces Mol-LlaMA, a multimodal LLM designed for general understanding and reasoning about molecules. The authors create a novel instruction dataset with three types of synthetic data: structural descriptions of functional groups, structure-to-feature relationship explanations, and conversations based on the first two, all generated using GPT-4o.\n\nArchitecture-wise, Mol-LlaMA uses 2D (MoleculeSTM) and 3D (UniMol) molecular encoders to capture structural information. These embeddings are aligned through a blending module, and a Q-Former projects the combined embeddings to the LLM.\n\nDetailed experimental results show that Mol-LlaMA outperforms other baselines across various tasks.", "questions": "* Is there any evaluation performed on GPT-4o's accuracy in extracting functional groups and their connectivity information from the IUPAC notation of the molecule when generating the \"Detailed Structural Description\" data type?\n\n* Please include a full ablation study on other benchmarks like property prediction, MoleculeQA, and 3D-MoIT? This could help to better understand the comprehensive benefits of the Mol-LLaMA architecture and data types.\n\n* Is there any plan to publicly release the data and code?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n* The authors present a novel and comprehensive instruction dataset that covers structural, chemical, and biological reasoning data about the molecule.\n* The proposed 2D-3D blending module effectively aligns information from different molecular encoders, and the benefits of this over single-encoder approaches are covered in the experimental results.\n* Qualitative and quantitative experimental results show that Mol-LlaMA outperforms baselines on tasks like property prediction and molecular comprehension.\n\n\n**Weaknesses:**\n\n* The instruction dataset is fully synthetic and heavily relies on GPT-4o. Although GPT-4o is used to filter incorrect data, some hallucinations may still persist. A more rigorous evaluation of the dataset quality would strengthen the work.\n* GPT-4o is used both for generating and evaluating the data, which may introduce bias. Including human evaluations would provide a more robust and independent assessment of Mol-LlaMA’s capabilities.", "was_revised": true, "final_justification": "The authors present a novel method to construct an instruction dataset for molecules, combining structural, chemical, and biological reasoning data. The experimental results are promising and highlight the potential of the method. \n\nWhile some of my concerns about the evaluation with GPT-4o were addressed, the issues of dataset quality and potential hallucinations due to reliance on IUPAC naming remain unresolved.", "rebuttal": "We sincerely thank you for your constructive and helpful comments. We appreciate the positive comments.\n- The proposed instruction dataset is novel and comprehensive, covering structural, chemical, and biological reasoning data.\n- The proposed blending module effectively aligns information from different molecular encoders.\n- Experimental results show that Mol-LLaMA outperforms baselines.\n\nWe initially address your concerns below.\n\n---\n*Comment 1*: The dataset is fully synthetic, relying on GPT-4o.\n\n*Response 1*: To avoid potential misunderstanding, we would like to clarify that, while our dataset involves synthetic generation with GPT-4o, we mitigate possible hallucinations by grounding molecular feature annotations and IUPAC names using PubChem. This grounding ensures alignment with established chemical knowledge and enhances the reliability of the dataset.\n\n---\n*Comment 2*: A more rigorous evaluation of the dataset quality beyond using GPT-4o would strengthen the work.\n\n*Response 2*: We appreciate your valuable suggestions. However, we chose GPT-4o as the primary model because, to the best of our knowledge, **it is currently the only LLM whose capabilities in molecular and scientific reasoning have been extensively analyzed and documented** [1]. At the time of our study, other LLMs lacked comparable in-depth evaluations in the molecular domain, which limited the reliability for those LLMs.\n\nTo validate whether other LLMs actually help to improve the dataset quality, we further filter out data using diverse LLMs: GPT-4o, Qwne3-14B, Gemma-3-12B-IT, and Llama-3.1-8B-Instruct, and select the data with a score of 4 for all evaluator LLMs. As shown in Table R1, the quadruple filtration (Mol-LLaMA$^*$) does not bring a significant performance improvement, implying that other LLMs misjudge to remove the correct data and narrow the knowledge scope of the trained model. We will add the experimental results in the final revision.\n\nEven though the current LLMs have limitations in evaluating data quality, **our dataset construction pipeline is readily adaptable with emerging LLMs that exhibit strong domain expertise in molecular sciences, which is a notable contribution for building molecular LLMs**.\n\n|Models|Help.|Relev.|Acc.|Details|Overall|Help.|Relev.|Acc.|Details|Overall|Help.|Relev.|Acc.|Details|Overall|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|Mol-LLaMA|1.126|1.145|1.154|1.090|1.125|1.224|1.266|1.302|1.211|1.251|1.578|1.840|2.030|1.528|1.744|\n|Mol-LLaMA$^*$|1.004|1.024|1.035|0.941|1.006|1.143|1.193|1.255|1.113|1.179|1.605|1.858|2.097|1.569|1.801|\n\n*Table R1: Quantitative evaluation on structural (left 5 criteria), chemical (middle 5 criteria), and biological (right 5 criteria) understanding.*\n\n[1] Microsoft Research AI4Science, and Microsoft Azure Quantum. \"The impact of large language models on scientific discovery: a preliminary study using gpt-4.\" arXiv preprint arXiv:2311.07361 (2023).\n\n---\n*Comment 3*: GPT-4o is used both for data generation and performance evaluation, which may introduce bias.\n\n*Response 3*: Thank you for your valuable suggestion. To robustly evaluate the quality of generated responses, we report the average of all metrics measured by GPT-4o, Qwen3-14B, Gemma-3-12B-IT, and Llama-3.1-8B-Instruct. As shown in Table R2, **Mol-LLaMA continues to outperform all baselines, including GPT-4o, and the overall trend remains consistent with Table 3, indicating that the evaluation of other LLMs agree with the one of GPT-4o**. We will include these results in the final revision.\n\n|Models|Help.|Relev.|Acc.|Details|Overall|Help.|Relev.|Acc.|Details|Overall|Help.|Relev.|Acc.|Details|Overall|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|**Llama-2-based**|\n||\n|Llama-2-7B-Chat|0.372|0.374|0.264|0.359|0.328|0.534|0.520|0.405|0.551|0.492|0.479|0.408|0.345|0.598|0.441|\n|Mol-Instructions|0.217|0.239|0.252|0.131|0.202|0.242|0.274|0.301|0.145|0.229|0.319|0.376|0.416|0.201|0.315|\n|LlasMol|0.278|0.306|0.304|0.204|0.265|0.295|0.335|0.312|0.215|0.280|0.332|0.378|0.443|0.260|0.341|\n|3D-MoLM|0.674|0.658|0.564|0.719|0.643|0.784|0.788|0.701|0.798|0.762|0.898|0.945|0.953|0.911|0.923|\n|LLaMo|0.306|0.407|0.443|0.179|0.312|0.345|0.464|0.524|0.197|0.356|0.454|0.630|0.802|0.236|0.490|\n|Mol-LLaMA|1.042|1.063|1.063|0.960|1.038|1.124|1.179|1.212|1.044|1.146|1.376|1.607|1.683|1.167|1.468|\n||\n|**Llama3.1-based**|\n||\n|Llama3.1-8B|0.669|0.680|0.560|0.648|0.634|0.698|0.693|0.600|0.695|0.667|0.705|0.663|0.617|0.732|0.674|\n|Mol-Instructions|0.261|0.334|0.369|0.152|0.263|0.265|0.350|0.396|0.149|0.269|0.358|0.468|0.577|0.191|0.367|\n|3D-MoLM|0.835|0.874|0.794|0.782|0.817|0.902|0.973|0.902|0.829|0.900|1.114|1.295|1.346|0.970|1.180|\n|LLaMo|0.480|0.616|0.560|0.301|0.473|0.401|0.534|0.560|0.245|0.414|0.596|0.788|0.834|0.311|0.603|\n|Mol-LLaMA|1.052|1.081|1.092|0.964|1.049|1.139|1.192|1.225|1.068|1.163|1.459|1.699|1.801|1.210|1.555|\n|Mol-LLaMA$^*$|1.001|1.026|1.030|0.911|0.997|1.113|1.170|1.205|1.034|1.139|1.421|1.640|1.741|1.223|1.522|\n\n*Table R2:  Quantitative evaluation on structural (left 5 criteria), chemical (middle 5 criteria), and biological (right 5 criteria) understanding using diverse evaluators.*\n \n---\n*Comment 4*: Including human evaluations would provide a more robust and independent assessment of Mol-LlaMA’s capabilities.\n\n*Response 4*: Unfortunately, **conducting human evaluation is difficult due to the high complexity of accurately assessing molecular properties**. Accurate and reliable evaluations of model-generated responses in the scientific domain require deep interdisciplinary expertise spanning chemistry, biochemistry, biology, and pharmacology. Furthermore, models may generate correct responses that are not included in the annotated descriptions as shown in Table 11, potentially introducing evaluator bias and making an accurate assessment challenging. Consequently, gathering evaluators possessing the required interdisciplinary expertise poses significant practical difficulties.\n\nAs an alternative, **we rigorously evaluate Mol-LLaMA across diverse and independent tasks, providing a robust demonstration of its capabilities**. Specifically, Mol-LLaMA demonstrates its capabilities on understanding general molecular features including structural, chemical, and biological features (Tables 3 and R2 of *Response 3*), basic and quantum property prediction (Tables 4 and 13), specific biological property prediction (Table 8), and molecular comprehension benchmarks (Tables 5 and 12).\n\n---\n*Comment 5*: Is there any evaluation performed on GPT-4o's accuracy in extracting functional groups and their connectivity information from the IUPAC notation of the molecule when generating the \"Detailed Structural Description\" data type?\n\n*Response 5*: To the best of our knowledge, **there is currently no benchmark to evaluate the accuracy of understanding molecular structures with IUPAC names, as annotating IUPAC names requires specialized expertise**. Meanwhile, **we chose to use IUPAC names and GPT-4o based on the experimental analyses reported in the literature.** Microsoft Research AI4Science and Microsoft Azure Quantum teams [1] conducted extensive evaluation of GPT-4 within the scientific contexts, providing informative analyses that have not been conducted for other LLMs. Additionally, the work [1] has shown that IUPAC names are the most interpretable string representations, as their tokenization of IUPAC names provides subword semantics of molecular substructures. Therefore, our selection of both IUPAC names and GPT-4o is grounded in empirical evidence and scientific relevance.\n\n---\n*Comment 6*: A full ablation study on other benchmarks.\n\n*Response 6*: The primary aim of the ablation study on the PAMPA task is to evaluate the knowledge and reasoning capabilities required for predicting molecular properties. Therefore, we focused on data types that significantly influence these capabilities of LLMs.\n\nWe further conduct an ablation study on the task transfer scenario using MoleculeQA. As shown in Table R3, each data type contributes incrementally to performance with steady improvements observed across data types (i.e., S $\\rightarrow$ S+F $\\rightarrow$ Full). Table R4 further demonstrates that incorporating different molecular representations and using the blending module improves performance in total, suggesting that combining 2D and 3D representations is beneficial and that the blending module effectively integrates complementary information. Overall, the impact of data types on performance gains is greater than that of model architecture, highlighting the importance of instruction dataset quality in task transfer scenarios. We will add the experimental results in the final revision.\n\n|Method|Struct.|Source|Prop.|App.|Totals|\n|-|-|-|-|-|-|\n|S|72.74|71.41|49.36|47.27|66.84|\n|S+F|76.33|72.19|46.31|45.91|68.42|\n|Conv.|72.53|71.41|48.88|45.46|66.48|\n|Full.|77.81|75.50|49.63|49.30|70.76|\n\n*Table R3: Ablation study of data types on MoleculeQA*\n\n|Method|Struct.|Source|Prop.|App.|Totals|\n|-|-|-|-|-|-|\n|2D|72.26|73.79|49.59|47.75|67.21|\n|3D|76.20|74.42|50.94|50.45|69.93|\n|2D+3D(Concat)|76.77|75.61|49.56|48.58|70.14|\n|2D+3D(Blended)|77.81|75.50|49.63|49.30|70.76|\n\n*Table R4: Ablation study of blending module on MoleculeQA*\n\n---\n*Comment 7*: Is there any plan to publicly release the data and code?\n\n*Response 7*: We will publicly release all materials, including data, code, and model weights, to be broadly utilized for the scientific domain.\n\n---\nWe thank the reviewer for your time and feedback and hope that our responses have addressed your concerns. If you have any further questions, we are happy to address them. If your concerns are appropriately addressed, we hope you kindly consider updating the rating accordingly."}, {"review_id": "r06Ib3l5wo", "reviewer": "Reviewer_JRtJ", "summary": "In this work, the author proposed Mol-LLaMA which could poentially enhance the understanding of molecular features with strong reasoning and explainability. It introduces a novel instruction dataset that includes detailed structural descriptions, structure-to-feature relationship explanations, and comprehensive conversations. The model also integrates complementary information from different molecular encoders, improving the understanding of molecular structures and features. Overall, Mol-LLaMA outperforms existing models in predicting and explaining molecular properties, potentially accelerating scientific discovery in chemistry and biology.", "questions": "1. Use more nuanced evaluation criteria that consider both text similarity and chemical factual accuracy to provide a more balanced assessment of the model's performance.\n2. Include a wider variety of molecular structures and properties in the instruction dataset to ensure comprehensive coverage like QM9 [1] or GEOM datasets.[2]\n\n[1] Ramakrishnan R, Dral P O, Rupp M, et al. Quantum chemistry structures and properties of 134 kilo molecules[J]. Scientific data, 2014, 1(1): 1-7.\n\n[2] Axelrod S, Gomez-Bombarelli R. GEOM, energy-annotated molecular conformations for property prediction and molecular generation[J]. Scientific Data, 2022, 9(1): 185.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "1. Provided chemistry dataset includes detailed structural descriptions, structure-to-feature relationship explanations.\n2. The blending module integrates information from different molecular encoders, improving the accuracy of molecular analysis.\n3. Mol-LLaMA outperforms existing models in predicting and explaining molecular properties, showcasing its potential to accelerate scientific discovery in chemistry and biology.\n\nWeaknesses:\n1. There is a concern about whether LLMs are fundamentally suitable for certain tasks in the benchmark, such as molecular property prediction, which is why only LLM results were presented.\n2. The molecule-to-text evaluation relies exclusively on text similarity metrics (BLEU, ROUGE, METEOR), which are inadequate for assessing chemical factual accuracy.\n3. The evaluation of LLMs based on helpfulness and relevance scores raises doubts about whether these scores accurately reflect the correctness of the information.\n4. The multimodal model and multimodal text data for molecules appear to lack originality.\n\n[1] Liu P, Ren Y, Tao J, et al. Git-mol: A multi-modal large language model for molecular science with graph, image, and text[J]. Computers in biology and medicine, 2024, 171: 108073.\n[2] Liu S, Nie W, Wang C, et al. Multi-modal molecule structure–text model for text-based retrieval and editing[J]. Nature Machine Intelligence, 2023, 5(12): 1447-1457.", "was_revised": true, "final_justification": "The authors effectively addressed concerns regarding the suitability of LLMs for molecular property prediction by highlighting Mol-LLaMA’s strengths in interpretability, flexible interaction, and fine-tuning capabilities, supported by empirical results.", "rebuttal": "We sincerely thank you for your constructive and helpful comments. We appreciate the positive comments.\n- Constructed instruction dataset includes detailed structural descriptions and structure-to-feature relationship explanations.\n- The proposed blending module improves the accuracy of molecular analysis.\n- Mol-LLaMA outperforms in predicting and explaining molecular properties, showing its potential to accelerate scientific discovery.\n\nWe initially address your concerns below.\n\n---\n*Comment 1*: There is a concern about whether LLMs are fundamentally suitable for certain tasks in the benchmark, such as molecular property prediction.\n\n*Response 1*: LLMs and specifically Mol-LLaMA exhibit significant advantages for scientific domains in three key areas: 1) **Interpreability**, 2) **Flexible Interaction**, and 3) **Fine-tuning Capability**.\n\n> ### Interpretability\n\n**LLMs, including Mol-LLaMA, generate interpretable outputs by detailed explanations, which is critical for scientific applications such as drug discovery where reliability and transparency are essential**. In contrast, traditional predictive models lack interpretability, limiting their practical application. Especially, Mol-LLaMA exhibits its interpretability and reasoning capabilities for molecules, providing relevant, helpful, and detailed responses as shown in Table 3 and 4. This interpretability of Mol-LLaMA can accelerate scientific discovery by fostering trust and insight.\n\n> ### Flexible Interaction\n\n**LLMs and Mol-LLaMA support interactive and adaptive use cases, such as user-driven prompting and the integration of external domain knowledge. This flexibility is valuable for addressing complex scientific problems**. One might provide additional molecule- or task-specific information via prompts to guide the model predictions, enhancing both performance and interpretability. Especially, Mol-LLaMA is beneficial in this utilization as shown in Table 4. Specifically, Mol-LLaMA shows a significant performance gain and achieves the best performance when prompted with task-specific information while maintaining helpful and interpretable responses.\n\n> ### Fine-tuning Capability\n\n**Open-source LLMs can be specialized for specific tasks via fine-tuning. In Table 13, we demonstrate that Mol-LLaMA, fine-tuned on property prediction tasks, significantly outperforms the state-of-the-art model without LLMs** (UniMol). This result highlights the promising opportunities for further utilization of Mol-LLaMA via fine-tuning. Please note that all datasets, codes, and model weights will be publicly released.\n\n---\n*Comment 2*: The molecule-to-text evaluation relies exclusively on text similarity metrics (BLEU, ROUGE, METEOR), which are inadequate for assessing chemical factual accuracy.\n\n*Response 2*: **In the main experiments of Table 3 and 4, we do not use similarity metrics such as BLEU, ROUGE, or METEOR, but leverage the LLM-as-a-judge** to evaluate not only the factual accuracy but also the quality and informativeness by assessing the textual semantics via LLMs. **In Table 12, we strictly follow the evaluation settings of the benchmark to ensure fair comparison** with the baselines, showing the superior performance of our Mol-LLaMA in the task transfer scenarios.\n\n---\n*Comment 3*: The evaluation of LLMs based on helpfulness and relevance scores raises doubts about whether these scores accurately reflect the correctness of the information.\n\n*Response 3*: **The helpfulness and relevance scores depend on the correctness, since these metrics inherently consider the factual correctness**. We experimentally validate these characteristics by intentionally generating factually incorrect outputs. Specifically, given a pair of the $i$-th molecule $X\\_i$ and the generated structural description of the $i$-th molecule $D\\_i$, we instruct GPT-4o to evaluate the quality of a permuted pair, defined as $(X\\_i, D\\_j)$ where $i\\\\neq j$. As shown in Table R1, factually incorrect outputs (Permuted) significantly degrade the overall performance, including helpfulness and relevance, indicating that these metrics are affected by the factual correctness.\n\n||Help|Relev|Acc|Details|Overall|\n|-|-|-|-|-|-|\n|Original|1.126|1.145|1.154|1.090|1.125|\n|Permuted|0.353|0.299|0.219|0.346|0.300|\n\n*Table R1: Quantitative results on structural understanding*\n\n---\n*Comment 4*: The multimodal model and multimodal text data for molecules appear to lack originality. [1, 2]\n\n*Response 4*: Compared to the reference works, our work made significant contributions in threefolds: 1) **dataset construction pipeline tailored to molecules**, 2) **model architecture to effectively combine the different representations**, and 3) **application-level contributions**.\n\nFirst, **we propose a dataset construction pipeline for molecular understanding, which is not explored in the reference works**. We observe that relying on public databases, including the datasets used in [1, 2], limits the knowledge scope of trained models, often generates irrelevant responses, makes wrong predictions, and limits interpretability [lines 46-52]. To address this limitation, from our key insight into the hierarchical relationships between molecular structures and their properties [lines 132-136], we design three data types and construct a large and comprehensive instruction dataset. Our dataset not only facilitates an understanding of general molecular features but also enhances explainability and reasoning capabilities [lines 138-159], which brings a substantial contributions compared to the multimodal text data in [1, 2].\n\nSecond, **we propose a model architecture that effectively integrates the complementary information from different and essential molecular representations**. Specifically, the main contributions on the model architectures can be summarised as follows:\n- We utilize both 2D and 3D molecular representations as they have their distinct and essential information for modeling molecular structures [lines 167-172]. In contrast, the reference work [1] employs 2D molecular structures with the images of 2D molecular structures, not providing additional information from using images. \n- Our proposed blending module effectively integrates these distinct advantages between 2D and 3D molecular encoders [lines 173-178], while the reference work [1] utilizes a simple concatenation, which cannot fully leverage the complementary information from different molecular representations as experimentally validated in Table 6 (Right, 2D+3D (Concat)).\n\nThird, our work and multi-modal model, Mol-LLaMA, have distinct advantages for the practical utility as follows:\n- Model Abilities: **Mol-LLaMA can play diverse roles with users and, thus facilitate the scientific discovery, whereas the reference work [2] is specialized for several tasks** because of the limited model architecture. Please note the reference work [2] (i.e., MoleculeSTM) is one of our molecular encoders [lines 171-172].\n- Further Utilization: **Our dataset construction pipelines and model architecture can be further utilized for other biomolecules**, including RNAs, proteins, and their complexes [lines 299-309], which are fundamental and essential components to understand the living organisms.\n\n[1] Liu, Pengfei, et al. \"Git-mol: A multi-modal large language model for molecular science with graph, image, and text.\" Computers in biology and medicine 171 (2024): 108073.\n\n[2] Liu, Shengchao, et al. \"Multi-modal molecule structure–text model for text-based retrieval and editing.\" Nature Machine Intelligence 5.12 (2023): 1447-1457.\n\n---\n*Comment 5*: Include a wider variety of molecular structures and properties in the instruction dataset to ensure comprehensive coverage like QM9 [3] or GEOM datasets. [4]\n\n*Response 5*: Our dataset includes 320K molecular structures derived from PubChem, providing **extensive coverage comparable to QM9 (133K) and GEOM (450K)**. However, **unlike QM9 and GEOM, which primarily annotate highly specific numerical properties** (e.g., quantum chemical properties in QM9 and BBBP or BACE-1 binding affinity in GEOM), **our dataset focuses on general molecular properties described textually**. The narrow and numeric-specific annotations in QM9 and GEOM limit their applicability for instructional purposes, as discussed in lines 118-125. By leveraging PubChem’s broad textual descriptions, our dataset effectively captures a comprehensive spectrum of molecular properties and ensures suitability for diverse instructional tasks.\n\n[3] Ramakrishnan, Raghunathan, et al. \"Quantum chemistry structures and properties of 134 kilo molecules.\" Scientific data 1.1 (2014): 1-7.\n\n[4]  Axelrod, Simon, and Rafael Gomez-Bombarelli. \"GEOM, energy-annotated molecular conformations for property prediction and molecular generation.\" Scientific Data 9.1 (2022): 185.\n\n---\nWe thank the reviewer for your time and feedback and hope that our responses have addressed your concerns. If you have any further questions, we are happy to address them. If your concerns are appropriately addressed, we hope you kindly consider updating the rating accordingly."}, {"review_id": "2clmKb5LFp", "reviewer": "Reviewer_6V81", "summary": "In this paper, Mol-LLaMA, a large molecular language model that grasps the general knowledge centered on molecules and exhibits explainability and reasoning ability, is introduced. To develop the model, a novel, large-scale instruction dataset (Mol-LLaMA-Instruct) is constructed with GPT-4o, featuring three specific data types designed to teach foundational molecular knowledge and causal relationships. To improve the performance, the authors design a \"blending module\" that integrates information from both 2D and 3D molecular encoders to create a more comprehensive structural representation.  The paper demonstrates through extensive experiments that Mol-LLaMA outperforms existing models, including GPT-4o, in tasks such as molecular understanding and property prediction.", "questions": "1. How effective are the molecular representations learned by the proposed model for direct application in downstream tasks? Furthermore, what are its specific advantages over previous unified representation models, such as in [1]?\n2. Regarding the evaluation methodology using GPT-4o as a judge: The paper describes a pairwise comparison to derive a relative score. Was one of the two assistants consistently the GPT-4o model itself, acting as a reference baseline? The prompt instructs the judge to avoid order effects, but were procedural safeguards, such as randomizing the response order, implemented to mitigate positional bias? Finally, considering these factors, how robust is this relative scoring method compared to an absolute grading system where each response would be evaluated individually against a fixed rubric?\n3. Prompt Specificity and Bias: The model was trained and tested on a specific set of instruction formats. To what extent does this in-distribution testing create an advantage for Mol-LLaMA over other models that were not trained on these specific prompts?\n4. Zero-Shot vs. Fine-Tuning: On the MoleculeQA benchmark, all models were fine-tuned. Would a zero-shot evaluation on this task be possible, and what might it reveal about the model's intrinsic generalization capabilities without task-specific training?\n5. Property Prediction Benchmarking: For the molecular property prediction tasks, the comparison could be expanded since Mol-LLaMA is finetuned on the dataset. Have you considered: Including more datasets from the same class (e.g., Tox21, HIV)? Benchmarking against smaller, state-of-the-art models that are specialized for these tasks such as [2] ?\n6. Using the Area Under the ROC Curve (AUROC) metric in addition to accuracy, as it can be more informative for classification tasks?\n7. Evaluation of Baselines: In the property prediction results (Table 4), the \"Fidelity\" and \"Helpfulness\" scores are not reported for the GPT-4o baseline. Could you clarify why these metrics were omitted for the baseline model?\n8. Does the model support inputs involving multiple molecules at once? The general LLMs such as GPT-4o can process SMILES of multiple molecules. \n9. Influence of Molecular Conformation: Given the use of a 3D encoder, how sensitive is the model's performance to different input molecular conformations? How effectively does it handle molecules with high conformational flexibility?\n10. How well does the model follow instructions and generalize to tasks beyond those it was explicitly trained on?\n11. Practical Utility: As a general-purpose tool, what are some concrete examples of how Mol-LLaMA could assist chemists or biologists in their day-to-day research workflows?\n12. Comparison with Other Models: The current benchmarks include models of a similar scale. How would Mol-LLaMA be expected to perform against significantly larger foundation models such as GPT-4.5? Could a comparison be made against other existing general-purpose (multi-modal) chemical language models that were not included in this study, such as [3, 4]?\n13. Open-Source Plans: Are there plans to release the model weights, training code, and the Mol-LLaMA-Instruct dataset to the public to support reproducibility and further community research? I don’t find the links in this paper.\n\nI recognize that my previous questions cover a wide range of topics, and it may not be feasible to address every point exhaustively. Therefore, you can focus your response on clarifying and defending the primary contribution of your work. The key is to demonstrate the unique value proposition of Mol-LLaMA. For instance:\nIf the core contribution is superior molecular representation, please provide more targeted evidence to demonstrate this superiority over existing methods.\nIf the central claim is that Mol-LLaMA is a general-purpose foundational model for chemistry, its evaluation should include a broader set of tasks to prove its versatility and superiority over other general-purpose chemical models.\nIf, however, Mol-LLaMA excels as a specialized model for a particular task, the most convincing comparison would be against state-of-the-art (SOTA) models dedicated to that task.\nClearly articulating and substantiating the unique value would strengthen the paper and my assessment of it.\n\n[1]Zhu, Jinhua, et al. \"Unified 2d and 3d pre-training of molecular representations.\" Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2022.\n[2]Zhang, Zaixi, et al. \"Motif-based graph self-supervised learning for molecular property prediction.\" Advances in Neural Information Processing Systems 34 (2021): 15870-15882.\n[3]Zhang, Di, et al. \"Chemllm: A chemical large language model.\" arXiv preprint arXiv:2402.06852 (2024).\n[4]Li, Junxian, et al. \"Chemvlm: Exploring the power of multimodal large language models in chemistry area.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 1. 2025.", "limitations": "1. No Generation abilities: Mol-LLaMA is designed primarily for molecular analysis and understanding.  It currently lacks the capability to perform generative tasks, such as designing novel molecules with desired properties. The authors have to consider how this model can help the improvement of chemistry or biology, \n2. Dataset Filtering: While the instruction dataset was filtered for factual accuracy using GPT-4o as a judge, this process is not infallible.  There is still a possibility that incorrect or subtly flawed information remains in the samples used for training.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "strength\n1. Comprehensive Instruction Dataset: A core strength is the systematic design of the Mol-LLaMA-Instruct dataset. Instead of relying on existing task-specific datasets, the authors created three new data types: \"Detailed Structural Description,\" \"Structure-to-Feature Relationship Explanation,\" and \"Comprehensive Conversation.\"  This approach explicitly aims to build a foundational understanding of molecular principles and enhance the model's reasoning capabilities. \n2. Innovative Model Architecture: The proposed 2D-3D blending module is an architectural contribution.  Recognizing that 2D and 3D representations offer complementary information (e.g., bond connectivity vs. spatial arrangement), the model uses a cross-attention mechanism to integrate them, which is shown to reduce errors and improve the understanding of molecular structures compared to using a single encoder or simple concatenation. \n\nweakness\n1. Reliance on GPT-4o for Data and Evaluation: The instruction dataset was generated using GPT-4o, and the quality of the generated responses was also evaluated using GPT-4o as a judge.  This heavy reliance on a proprietary, closed-source model introduces potential issues: \nInherited Biases: Any factual errors, biases, or limitations present in GPT-4o could be propagated into the training data and, consequently, into Mol-LLaMA.\nEvaluation Circularity: Using the same family of models for both data generation and evaluation can lead to results that favor the style and knowledge base of that model, potentially inflating performance scores.\n2. Lack of Statistical Significance Testing: The authors state that due to high computational costs, they did not train models multiple times.  While understandable, this makes it harder to assess the robustness and generalizability of the performance gains. At least, the authors should use temperature sampling and test the results multiple times.", "was_revised": true, "final_justification": "I still hold the reservation that the proposed method may not be the most effective path toward creating a truly general-purpose foundational model for chemistry. In my view, more convincing routes toward that goal would likely involve continued pre-training on large-scale chemical and biological corpora or learning from real-world feedback. For this reason, I believe the long-term significance of this specific approach is somewhat limited.\n\nNevertheless, considering the strong performance demonstrated across multiple tasks in the manuscript, I raise my rating to 4.", "rebuttal": "We sincerely thank you for your constructive and helpful comments. We appreciate the positive comments.\n- The design of the three new data types and the resulting instruction dataset builds a foundational understanding of molecular principles and enhances the reasoning capabilities\n- The proposed 2D-3D blending module is innovative, reduces errors, and improves the understanding of molecular structures.\n\nDue to the character limit, we initially address your major concerns and some of your questions below.\n\n---\n*Comment 1*: The closed-source model (i.e., GPT-4o) could introduce potential issues: Factual errors or biases could be propagated into the training data.\n\n*Response 1*: We appreciate your valuable comment. **While we aimed to select reliable LLMs to ensure data quality, to the best of our knowledge, alternative models lack extensive analysis capabilities for molecular and scientific reasoning**. In other words, GPT-4o [1] remains the only model with demonstrated reliability in these domains, making it a more suitable choice despite its closed-source nature. \n\nFurther, we investigate the effect of other LLMs for evaluating the data quality by filtering with four LLMs: GPT-4o, Qwen1.5-14B, Gemma-1.5-12B-IT, and Llama-3.1-8B-Instruct, selecting samples rated 4 by all. As shown in Table R1, this quadruple-filtered version (Mol-LLaMA$^*$) yielded no significant gains, suggesting these LLMs may wrongly discard correct data and limit knowledge scope. We will include the results in the final revision.\n\nDespite the limitations of current LLMs, we would like to emphasize that **our dataset construction pipeline is a notable contribution for building molecular LLMs as it can be readily applied with upcoming LLMs exhibiting more extensive knowledge for scientific domains**.\n\n|Models|Help|Relev|Acc|Details|Overall|Help|Relev|Acc|Details|Overall|Help|Relev|Acc|Details|Overall|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|Mol-LLaMA|1.126|1.145|1.154|1.090|1.125|1.224|1.266|1.302|1.211|1.251|1.578|1.840|2.030|1.528|1.744|\n|Mol-LLaMA$^*$|1.004|1.024|1.035|0.941|1.006|1.143|1.193|1.255|1.113|1.179|1.605|1.858|2.097|1.569|1.801|\n\n*Table R1: Quantitative evaluation on structural (left 5 criteria), chemical (middle 5 criteria), and biological (right 5 criteria) understanding.*\n\n[1] Microsoft Research AI4Science and Microsoft Azure Quantum. \"The impact of large language models on scientific discovery: a preliminary study using gpt-4.\" arXiv:2311.07361 (2023).\n\n---\n*Comment 2*: Using the same family of models for both data generation and evaluation can lead to results that favor their style and knowledge base, potentially inflating performance scores.\n\n*Response 2*: We thank you for your valuable comment. To investigate whether the same family favors the style and knowledge base, we conduct the evaluation using four LLMs: GPT-4o, Qwen3-14B, Gemma-3-12B-IT, and Llama-3.1-8B-Instruct. As shown in Table R2, **Mol-LLaMA still outperforms all baselines, including GPT-4o, with consistent trends observed in Table 3, implying that GPT-4o’s evaluation is not biased rather agrees with other LLMs**. We will include these results in the final revision.\n\n|Models|Help|Relev|Acc|Details|Overall|Help|Relev|Acc|Details|Overall|Help|Relev|Acc|Details|Overall|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|**Llama2-based**|\n||\n|Llama-2-7B-Chat|0.372|0.374|0.264|0.359|0.328|0.534|0.520|0.405|0.551|0.492|0.479|0.408|0.345|0.598|0.441|\n|Mol-Instructions|0.217|0.239|0.252|0.131|0.202|0.242|0.274|0.301|0.145|0.229|0.319|0.376|0.416|0.201|0.315|\n|LlasMol|0.278|0.306|0.304|0.204|0.265|0.295|0.335|0.312|0.215|0.280|0.332|0.378|0.443|0.260|0.341|\n|3D-MoLM|0.674|0.658|0.564|0.719|0.643|0.784|0.788|0.701|0.798|0.762|0.898|0.945|0.953|0.911|0.923|\n|LLaMo|0.306|0.407|0.443|0.179|0.312|0.345|0.464|0.524|0.197|0.356|0.454|0.630|0.802|0.236|0.490|\n|Mol-LLaMA|1.042|1.063|1.063|0.960|1.038|1.124|1.179|1.212|1.044|1.146|1.376|1.607|1.683|1.167|1.468|\n||\n|**Llama3.1-based**|\n||\n|Llama3.1-8B|0.669|0.680|0.560|0.648|0.634|0.698|0.693|0.600|0.695|0.667|0.705|0.663|0.617|0.732|0.674|\n|Mol-Instructions|0.261|0.334|0.369|0.152|0.263|0.265|0.350|0.396|0.149|0.269|0.358|0.468|0.577|0.191|0.367|\n|3D-MoLM|0.835|0.874|0.794|0.782|0.817|0.902|0.973|0.902|0.829|0.900|1.114|1.295|1.346|0.970|1.180|\n|LLaMo|0.480|0.616|0.560|0.301|0.473|0.401|0.534|0.560|0.245|0.414|0.596|0.788|0.834|0.311|0.603|\n|Mol-LLaMA|1.052|1.081|1.092|0.964|1.049|1.139|1.192|1.225|1.068|1.163|1.459|1.699|1.801|1.210|1.555|\n|Mol-LLaMA$^*$|1.001|1.026|1.030|0.911|0.997|1.113|1.170|1.205|1.034|1.139|1.421|1.640|1.741|1.223|1.522|\n\n*Table R2:  Quantitative evaluation on structural (left 5 criteria), chemical (middle 5 criteria), and biological (right 5 criteria) understanding using diverse evaluators.*\n\n---\n*Comment 3*: Lack of statistical significance testing makes it harder to assess the robustness and generalizability.\n\n*Response 3*: We appreciate your valuable suggestion. However, **we adopt greedy decoding, as it aligns with real-world applications where users prioritize the most confident and reliable responses from LLMs, specifically for the scientific problems**. Instaed, we perform the LLM evaluation three times to assess the quality of the generated responses.\n\nFurthermore, we apply the temperature sampling with three runs and report the averaged metrics in Table R3 using the same four evaluator LLMs in *Response 2*. **Performance is maintained and Mol-LLaMA consistently outperforms the baselines, including GPT-4o, indicating that Mol-LLaMA robustly and generally shows superior performance**.\n\n|Models|Help|Relev|Acc|Details|Overall|Help|Relev|Acc|Details|Overall|Help|Relev|Acc|Details|Overall|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|Llama-3.1-8B-Instruct|0.650|0.671|0.555|0.631|0.623|0.699|0.691|0.619|0.719|0.675|0.724|0.704|0.657|0.761|0.706|\n|Mol-Instructions|0.258|0.327|0.355|0.149|0.258|0.275|0.345|0.391|0.153|0.275|0.355|0.456|0.583|0.190|0.370|\n|3D-MoLM|0.694|0.679|0.575|0.734|0.663|0.756|0.761|0.665|0.771|0.731|0.869|0.886|0.852|0.874|0.870|\n|LLaMo|0.475|0.592|0.543|0.306|0.465|0.405|0.526|0.554|0.241|0.413|0.595|0.746|0.795|0.311|0.588|\n|Mol-LLaMA|1.035|1.046|1.068|0.993|1.040|1.118|1.164|1.202|1.066|1.140|1.390|1.588|1.693|1.206|1.483|\n\n*Table R3: Quantitative evaluation on structural (left 5 criteria), chemical (middle 5 criteria), and biological (right 5 criteria) understanding with temperature sampling*\n\n---\n*Comment 4*: As a general-purpose tool, what are some concrete examples of how Mol-LLaMA could assist chemists or biologists in their research workflows?\n\n*Response 4*: **We carefully designed the experiments to show the effectiveness of Mol-LLaMA in scientific research workflows**, focusing on three key points: **1) Molecular analysis**, **2) Interactive manipulation**, and **3) Task specialization**.\n\n> ### Analysis of De novo Molecules\n\nChemists, biologists, and drug developers frequently encounter novel molecules that lack annotations in public databases. In such cases, Mol-LLaMA enables initial assessments by providing insights into physicochemical properties and likely biological functions. Tables 3 and 4 demonstrate Mol-LLaMA’s effectiveness in this utilization, showcasing its superior ability in accurately predicting molecular features with helpful explanations.\n\n> ### Interactive Manipulation\n\nAnother key strength of Mol-LLaMA is its interactive capability, allowing users to guide its outputs through prompts. For the PAMPA task, we devise “with task info” prompting that injects the task-specific information reflecting real-world scenarios where users guide LLM outputs. As shown in Table 4, Mol-LLaMA built upon LLaMA3.1, shows performance gains when prompted with task-specific information, demonstrating its ability to effectively handle diverse user prompts.\n\n> ### Task Specialization\n\nMol-LLaMA can be further fine-tuned for specific tasks, making it well-suited for researchers and developers working in highly specialized domains such as drug discovery. We validate Mol-LLaMA’s effectiveness in task transfer settings of Tables 5, 12, and 13, demonstrating superior performance compared to baseline models.\n\n---\n*Comment 5*: To what extent does this in-distribution testing create an advantage for Mol-LLaMA over other models that were not trained on these specific prompts?\n\n*Response 5*: **In the PAMPA task, we assess Mol-LLaMA’s generalization ability using prompts that were entirely different from those seen during training**, as shown in Table 32. These settings show that Mol-LLaMA is capable of reasoning and generating accurate answers beyond the training prompts, showcasing its generalization ability. \n\n---\n*Comment 6*: Have you considered more datasets such as MoleculeNet?\n\n*Response 6*: We opted for the PAMPA task as it is well-suited to evaluate whether molecular LLMs truly understand molecular structures and features. In contrast, tasks in MoleculeNet require external knowledge beyond the molecule itself such as the structure of the blood-brain barrier for the BBBP task or nuclear receptors for the Tox21 task which heavily influenced by the internal knowledge of the base LLM, making fair comparisons difficult.\n\nNevertheless, we include a comparison on the BBBP task from MoleculeNet in Table 8, where Mol-LLaMA outperforms existing molecular LLMs built upon the same model architectures.\n\n---\n*Comment 7*: Are there plans to release the model weights, training code, and the Mol-LLaMA-Instruct dataset?\n\n*Response 7*: We will release the Mol-LLaMA weights, training code, and Mol-LLaMA-Instruct dataset to facilitate further research in molecular foundation models and practical usage for the scientific domain.\n\n---\nWe thank the reviewer for your time and feedback and hope that our responses have addressed your concerns. If you have any further questions, we are happy to address them. If your concerns are appropriately addressed, we hope you kindly consider updating the rating accordingly."}, {"review_id": "i1efnFHpMX", "reviewer": "Reviewer_voqH", "summary": "This paper presents Mol-LLaMA, a general-purpose molecular assistant designed to perform molecular analysis with enhanced reasoning and explainability. The primary contribution is the creation of Mol-LLaMA-Instruct, a large instruction-following dataset generated via GPT-4o, incorporating hierarchical reasoning across structural, chemical, and biological levels. The dataset construction leverages annotated IUPAC names and curated PubChem descriptions, using GPT-4o to generate multi-level reasoning samples.\n\nThe model architecture integrates 2D and 3D molecular encoders, a 2D–3D blending module, and a Q-Former to project molecular features into a large language model space. The authors adopt a two-stage training procedure involving molecular representation learning followed by multi-modal instruction tuning. Evaluation is conducted using GPT-4o as an automated judge across multiple criteria such as helpfulness, relevance, and accuracy.", "questions": "How well justified is the choice of molecular encoders (MoleculeSTM for 2D, UniMol for 3D)? Were alternative encoders considered or compared?", "limitations": "I did not observe any major formatting issues during my review of the paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n- The hierarchical design of the instruction dataset is well thought out and supports improved reasoning capabilities.\n- The integration of 2D and 3D molecular representations via a blending module effectively captures both topological and spatial features of molecules. This design choice is supported by ablation studies.\n\n**Weaknesses**\n\n- The paper primarily reuses existing tools such as GPT-4o for data generation and filtering, pre-trained molecular encoders, and Q-Former. The main novelty lies more in the engineering effort and the construction of the dataset rather than in introducing new models or learning algorithms.\n- The paper uses GPT-4o both to generate molecular reasoning data and to evaluate the factual accuracy of the generated samples. While this is a practical approach given GPT-4o’s strong domain knowledge, I am concerned about potential risks of hallucination. If GPT-4o produces factually incorrect or misleading descriptions during generation, it may not always be able to reliably identify these errors during the evaluation phase, since both tasks rely on the same model’s internal knowledge. I suggest the authors discuss this limitation explicitly and, if possible, consider additional validation strategies, such as incorporating human expert review.\n- The instruction dataset is a central contribution of the paper. However, the details of how this dataset is constructed are largely placed in the appendix. I recommend moving the dataset construction process into the main text and expanding the discussion to make the contribution clearer and more accessible to readers.\n\n- *Minors:* The paper refers to the \"string version of the molecule\" and clarifies that this corresponds to the IUPAC name. Since IUPAC names are less commonly used as string representations in computational chemistry compared to formats like SMILES, SELFIES, or InChI, it would be clearer and more precise for the authors to explicitly state \"IUPAC name\" instead of the more ambiguous \"string version.\" This would help avoid confusion for readers familiar with standard molecular string formats.", "was_revised": true, "final_justification": "While I still believe the model-side method lacks novelty, I acknowledge that the newly introduced dataset is valuable. The inclusion of preliminary human evaluation results with a fairly high R2 is promising. Given this, I am willing to raise my rating, as I believe this work could make a useful contribution to the research community.", "rebuttal": "We sincerely thank you for your constructive and helpful comments. We appreciate the positive comments.\n- The hierarchical design of the dataset is well thought out and supports improved reasoning capabilities.\n- The integration of 2D and 3D molecular representations via a blending module effectively captures both topological and spatial features.\n\nWe initially address your concerns below.\n\n---\n*Comment 1*: The paper primarily reuses existing tools such as GPT-4o for data generation and filtering, pre-trained molecular encoders, and Q-Former. The main novelty lies more in the engineering effort and the construction of the dataset rather than in introducing new models or learning algorithms.\n\n*Response 1*: We propose a novel framework to build molecular LLMs, making notable contributions in fourfold: 1) **novel dataset construction pipeline for molecules**, 2) **model architecture**, 3) **resulting foundational model, Mol-LLaMA**, and 4) **further utilization**.\n\n> ### Dataset Construction Pipeline\n- Motivation-level: **We first tackle that existing databses and dataset construction pipelines are not appropriate to build the molecular LLMs**, identifying the following two challenges: 1) molecular LLMs should learn a wide-ranging knowledge encompassing chemical and biological features as this knowledge is domain-specific and 2) unlike other modalities, molecular features are not directly apparent from structures, requiring complex reasoning and interpretability [lines 31-52, 112-125, and Table 2].\n- Methodology-level: To solve these limitations, **we design novel data types that have not been explored in the literature**, based on our observation of the hierarchical relationships between molecular structures and their features.\n\n> ### Model Architecture\n- Motivation-level: **We tackle that the current molecular LLMs rely on a single type of molecular representations**, often failing to correctly predict the molecular properties [lines 52-54].\n- Methodology-level: We observe that 2D and 3D molecular encoders have their distinct advantages. Thus, **we opt to utilize both molecular representations [lines 167-172] and propose an effective model architecture, the blending module, that can integrate their advantages [lines 173-178], which have not been covered by previous works.**\n\n> ### Resulting Foundational Model: Mol-LLaMA\n\nOur resulting model, Mol-LLaMA, exhibits several key strengths as follows:\n- General understanding of molecular features: **Mol-LLaMA accurately predicts the general molecular features compared to other molecular LLMs** as shown in Table 3 and 4, showcasing its potential as a general-purpose assistant for molecular analysis.\n- Explainability and Reasoning Capabilities: Unlike other molecular LLMs, **Mol-LLaMA provides interpretable responses with informative reasoning processes**  as shown in Table 1, 9, 10 11, and 17, enhancing the reliability and thus further opening its utilization for practical fields.\n\n> ### Further Utilization\n\nWe would like to emphasize that **our dataset construction pipeline and the model architecture can be further utilized to build other LLMs specialized in understanding the biomolecules such as proteins, RNAs and their complexes, which are notable contributions for the AI4Science field** [lines 306-309]. Please note that we will publicly release all materials, including code, data, and model weights, as a state-of-the-art foundational model for molecular fields.\n\n---\n*Comment 2*: While this is a practical approach given GPT-4o’s strong domain knowledge, I am concerned about potential risks of hallucination.\n\n*Response 2*: We appreciate your valuable comment. First, we would like to clarify that **our instruction dataset generation is grounded in the annotated molecular features and IUPAC names provided by PuChem, which helps mitigate the risk of hallucination**. \n\nAdditionally, **we adopt GPT-4o due to its demonstrated capabilities in scientific domains, as evidenced by prior studies [1], which support its reliability in this context**. In contrast, other LLMs have not been rigorously analyzed in the molecular science domain, making them less reliable for generating and evaluating data.\n\nTo show the effect of other LLMs, we filtered data using Qwen1.5-14B, Gemma-1.5-12B-IT, Llama-3.1-8B-Instruct and GPT-4o, selecting samples rated 4 by all. As shown in Table R1, this quadruple-filtered version (Mol-LLaMA$^*$) yielded no significant gains, suggesting these LLMs may wrongly discard correct data and limit model scope. We will include the results in the final revision.\n\nAlthough current LLMs are still limited, we would like to emphsize that **our dataset construction pipeline offers a notable contribution for building molecular LLMs by being readily adapted to future LLMs with more extensive knowledge in scientific domains**.\n\n\n|Models|Help.|Relev.|Acc.|Details|Overall|Help.|Relev.|Acc.|Details|Overall|Help.|Relev.|Acc.|Details|Overall|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|Mol-LLaMA|1.126|1.145|1.154|1.090|1.125|1.224|1.266|1.302|1.211|1.251|1.578|1.840|2.030|1.528|1.744|\n|Mol-LLaMA$^*$|1.004|1.024|1.035|0.941|1.006|1.143|1.193|1.255|1.113|1.179|1.605|1.858|2.097|1.569|1.801|\n\n*Table R1: Quantitative evaluation on structural (left 5 criteria), chemical (middle 5 criteria), and biological (right 5 criteria) understanding.*\n\n[1] AI4Science, Microsoft Research, and Microsoft Azure Quantum. \"The impact of large language models on scientific discovery: a preliminary study using gpt-4.\" arXiv preprint arXiv:2311.07361 (2023).\n\n---\n*Comment 3*: If possible, consider additional validation strategies, such as incorporating human expert review.\n\n*Response 3*: We thank you for your valuable suggestion. However, **gathering a sufficient number of qualified experts for accurately evaluating data quality is highly challenging**. First, assessing the generated molecular descriptions requires interdisciplinary expertise across chemistry, biochemistry, biology, and pharmacology. Additionally, some properties may not be annotated in public databases like PubChem, meaning factually correct descriptions could appear unsupported. The inherent ambiguity in certain molecular properties or descriptions may also introduce evaluator bias, as experts from different backgrounds might interpret the same description differently.\n\n---\n*Comment 4*: I recommend moving the dataset construction process into the main text and expanding the discussion to make the contribution clearer and more accessible to readers.\n\n*Response 4*: Due to the page limit, we explained the core components of our dataset construction pipelines, including context types, data types, and filtering process, postponing the details such as prompts to the appendix. We will move detailed explanations of the dataset construction pipeline to the main paper in the final revision.\n\n---\n*Comment 5*: It would be clearer and more precise for the authors to explicitly state \"IUPAC name\" instead of the more ambiguous \"string version.\"\n\nResponse 5. We will revise the term “string representation” to its specific type, such as SMILES or IUPAC name for clarity.\n\n---\n*Comment 6*: How well justified is the choice of molecular encoders (MoleculeSTM for 2D, UniMol for 3D)? Were alternative encoders considered or compared?\n\n*Response 6*: We adopt MoleculeSTM as it inherently captures molecular semantics through contrastive learning between 2D molecular structures and textual descriptions. Prior studies [2] have shown that contrastive learning with textual descriptions shows better performance than reconstruction-based training for building multimodal LLMs.\n\nUniMol is a state-of-the-art molecular encoder across both 2D and 3D representations. Although it is trained with a reconstruction loss, its use of transformer architectures enables strong representational expressiveness.\n\n[2] McKinzie, Brandon, et al. \"Mm1: methods, analysis and insights from multimodal llm pre-training.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.\n\n---\nWe thank the reviewer for your time and feedback and hope that our responses have addressed your concerns. If you have any further questions, we are happy to address them. If your concerns are appropriately addressed, we hope you kindly consider updating the rating accordingly."}]

Author rebuttal:

Meta-review: The paper introduces Mol-LLaMA, a large molecular language model designed to advance general molecular understanding through a novel instruction dataset and a multimodal architecture integrating 2D and 3D molecular encoders.  Reviewers generally agree on the strengths of the work, including the comprehensiveness and novelty of the dataset, the effectiveness of the proposed blending module, and the model’s strong empirical performance across multiple benchmarks. While some concerns were raised regarding reliance on GPT-4o for both data generation and evaluation, the authors provided extensive rebuttals, including additional experiments, multi-model comparisons, and limited human evaluation, which addressed most of the critical points.
Decision: Accept (poster)