Title: Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Molecular Language Model

General-purpose Assistant for Molecular Analysis

Molecular Comprehension
Abstract: Understanding molecules is key to understanding organisms and driving advances in drug discovery, requiring interdisciplinary knowledge across chemistry and biology. Although large molecular language models have achieved notable success in task transfer, they often struggle to accurately analyze molecular features due to limited knowledge and reasoning capabilities. To address this issue, we present Mol-LLaMA, a large molecular language model that grasps the general knowledge centered on molecules and exhibits explainability and reasoning ability. To this end, we design key data types that encompass the fundamental molecular features, taking into account the essential abilities for molecular reasoning. Further, to improve molecular understanding, we propose a module that integrates complementary information from different molecular encoders, leveraging the distinct advantages of molecular representations. Our experimental results demonstrate that Mol-LLaMA is capable of comprehending the general features of molecules and providing informative responses, implying its potential as a general-purpose assistant for molecular analysis. Our project page is at https://mol-llama.github.io/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Mol-LLaMA, a general-purpose molecular assistant designed to perform molecular analysis with enhanced reasoning and explainability. The primary contribution is the creation of Mol-LLaMA-Instruct, a large instruction-following dataset generated via GPT-4o, incorporating hierarchical reasoning across structural, chemical, and biological levels. The dataset construction leverages annotated IUPAC names and curated PubChem descriptions, using GPT-4o to generate multi-level reasoning samples.\n\nThe model architecture integrates 2D and 3D molecular encoders, a 2D–3D blending module, and a Q-Former to project molecular features into a large language model space. The authors adopt a two-stage training procedure involving molecular representation learning followed by multi-modal instruction tuning. Evaluation is conducted using GPT-4o as an automated judge across multiple criteria such as helpfulness, relevance, and accuracy.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The hierarchical design of the instruction dataset is well thought out and supports improved reasoning capabilities.\n- The integration of 2D and 3D molecular representations via a blending module effectively captures both topological and spatial features of molecules. This design choice is supported by ablation studies.\n\n**Weaknesses**\n\n- The paper primarily reuses existing tools such as GPT-4o for data generation and filtering, pre-trained molecular encoders, and Q-Former. The main novelty lies more in the engineering effort and the construction of the dataset rather than in introducing new models or learning algorithms.\n- The paper uses GPT-4o both to generate molecular reasoning data and to evaluate the factual accuracy of the generated samples. While this is a practical approach given GPT-4o’s strong domain knowledge, I am concerned about potential risks of hallucination. If GPT-4o produces factually incorrect or misleading descriptions during generation, it may not always be able to reliably identify these errors during the evaluation phase, since both tasks rely on the same model’s internal knowledge. I suggest the authors discuss this limitation explicitly and, if possible, consider additional validation strategies, such as incorporating human expert review.\n- The instruction dataset is a central contribution of the paper. However, the details of how this dataset is constructed are largely placed in the appendix. I recommend moving the dataset construction process into the main text and expanding the discussion to make the contribution clearer and more accessible to readers.\n\n- *Minors:* The paper refers to the \"string version of the molecule\" and clarifies that this corresponds to the IUPAC name. Since IUPAC names are less commonly used as string representations in computational chemistry compared to formats like SMILES, SELFIES, or InChI, it would be clearer and more precise for the authors to explicitly state \"IUPAC name\" instead of the more ambiguous \"string version.\" This would help avoid confusion for readers familiar with standard molecular string formats.\n\nQUESTIONS:\nHow well justified is the choice of molecular encoders (MoleculeSTM for 2D, UniMol for 3D)? Were alternative encoders considered or compared?\n\nLIMITATIONS:\nI did not observe any major formatting issues during my review of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, Mol-LLaMA, a large molecular language model that grasps the general knowledge centered on molecules and exhibits explainability and reasoning ability, is introduced. To develop the model, a novel, large-scale instruction dataset (Mol-LLaMA-Instruct) is constructed with GPT-4o, featuring three specific data types designed to teach foundational molecular knowledge and causal relationships. To improve the performance, the authors design a \"blending module\" that integrates information from both 2D and 3D molecular encoders to create a more comprehensive structural representation.  The paper demonstrates through extensive experiments that Mol-LLaMA outperforms existing models, including GPT-4o, in tasks such as molecular understanding and property prediction.\n\nSTRENGTHS AND WEAKNESSES:\nstrength\n1. Comprehensive Instruction Dataset: A core strength is the systematic design of the Mol-LLaMA-Instruct dataset. Instead of relying on existing task-specific datasets, the authors created three new data types: \"Detailed Structural Description,\" \"Structure-to-Feature Relationship Explanation,\" and \"Comprehensive Conversation.\"  This approach explicitly aims to build a foundational understanding of molecular principles and enhance the model's reasoning capabilities. \n2. Innovative Model Architecture: The proposed 2D-3D blending module is an architectural contribution.  Recognizing that 2D and 3D representations offer complementary information (e.g., bond connectivity vs. spatial arrangement), the model uses a cross-attention mechanism to integrate them, which is shown to reduce errors and improve the understanding of molecular structures compared to using a single encoder or simple concatenation. \n\nweakness\n1. Reliance on GPT-4o for Data and Evaluation: The instruction dataset was generated using GPT-4o, and the quality of the generated responses was also evaluated using GPT-4o as a judge.  This heavy reliance on a proprietary, closed-source model introduces potential issues: \nInherited Biases: Any factual errors, biases, or limitations present in GPT-4o could be propagated into the training data and, consequently, into Mol-LLaMA.\nEvaluation Circularity: Using the same family of models for both data generation and evaluation can lead to results that favor the style and knowledge base of that model, potentially inflating performance scores.\n2. Lack of Statistical Significance Testing: The authors state that due to high computational costs, they did not train models multiple times.  While understandable, this makes it harder to assess the robustness and generalizability of the performance gains. At least, the authors should use temperature sampling and test the results multiple times.\n\nQUESTIONS:\n1. How effective are the molecular representations learned by the proposed model for direct application in downstream tasks? Furthermore, what are its specific advantages over previous unified representation models, such as in [1]?\n2. Regarding the evaluation methodology using GPT-4o as a judge: The paper describes a pairwise comparison to derive a relative score. Was one of the two assistants consistently the GPT-4o model itself, acting as a reference baseline? The prompt instructs the judge to avoid order effects, but were procedural safeguards, such as randomizing the response order, implemented to mitigate positional bias? Finally, considering these factors, how robust is this relative scoring method compared to an absolute grading system where each response would be evaluated individually against a fixed rubric?\n3. Prompt Specificity and Bias: The model was trained and tested on a specific set of instruction formats. To what extent does this in-distribution testing create an advantage for Mol-LLaMA over other models that were not trained on these specific prompts?\n4. Zero-Shot vs. Fine-Tuning: On the MoleculeQA benchmark, all models were fine-tuned. Would a zero-shot evaluation on this task be possible, and what might it reveal about the model's intrinsic generalization capabilities without task-specific training?\n5. Property Prediction Benchmarking: For the molecular property prediction tasks, the comparison could be expanded since Mol-LLaMA is finetuned on the dataset. Have you considered: Including more datasets from the same class (e.g., Tox21, HIV)? Benchmarking against smaller, state-of-the-art models that are specialized for these tasks such as [2] ?\n6. Using the Area Under the ROC Curve (AUROC) metric in addition to accuracy, as it can be more informative for classification tasks?\n7. Evaluation of Baselines: In the property prediction results (Table 4), the \"Fidelity\" and \"Helpfulness\" scores are not reported for the GPT-4o baseline. Could you clarify why these metrics were omitted for the baseline model?\n8. Does the model support inputs involving multiple molecules at once? The general LLMs such as GPT-4o can process SMILES of multiple molecules. \n9. Influence of Molecular Conformation: Given the use of a 3D encoder, how sensitive is the model's performance to different input molecular conformations? How effectively does it handle molecules with high conformational flexibility?\n10. How well does the model follow instructions and generalize to tasks beyond those it was explicitly trained on?\n11. Practical Utility: As a general-purpose tool, what are some concrete examples of how Mol-LLaMA could assist chemists or biologists in their day-to-day research workflows?\n12. Comparison with Other Models: The current benchmarks include models of a similar scale. How would Mol-LLaMA be expected to perform against significantly larger foundation models such as GPT-4.5? Could a comparison be made against other existing general-purpose (multi-modal) chemical language models that were not included in this study, such as [3, 4]?\n13. Open-Source Plans: Are there plans to release the model weights, training code, and the Mol-LLaMA-Instruct dataset to the public to support reproducibility and further community research? I don’t find the links in this paper.\n\nI recognize that my previous questions cover a wide range of topics, and it may not be feasible to address every point exhaustively. Therefore, you can focus your response on clarifying and defending the primary contribution of your work. The key is to demonstrate the unique value proposition of Mol-LLaMA. For instance:\nIf the core contribution is superior molecular representation, please provide more targeted evidence to demonstrate this superiority over existing methods.\nIf the central claim is that Mol-LLaMA is a general-purpose foundational model for chemistry, its evaluation should include a broader set of tasks to prove its versatility and superiority over other general-purpose chemical models.\nIf, however, Mol-LLaMA excels as a specialized model for a particular task, the most convincing comparison would be against state-of-the-art (SOTA) models dedicated to that task.\nClearly articulating and substantiating the unique value would strengthen the paper and my assessment of it.\n\n[1]Zhu, Jinhua, et al. \"Unified 2d and 3d pre-training of molecular representations.\" Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2022.\n[2]Zhang, Zaixi, et al. \"Motif-based graph self-supervised learning for molecular property prediction.\" Advances in Neural Information Processing Systems 34 (2021): 15870-15882.\n[3]Zhang, Di, et al. \"Chemllm: A chemical large language model.\" arXiv preprint arXiv:2402.06852 (2024).\n[4]Li, Junxian, et al. \"Chemvlm: Exploring the power of multimodal large language models in chemistry area.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 1. 2025.\n\nLIMITATIONS:\n1. No Generation abilities: Mol-LLaMA is designed primarily for molecular analysis and understanding.  It currently lacks the capability to perform generative tasks, such as designing novel molecules with desired properties. The authors have to consider how this model can help the improvement of chemistry or biology, \n2. Dataset Filtering: While the instruction dataset was filtered for factual accuracy using GPT-4o as a judge, this process is not infallible.  There is still a possibility that incorrect or subtly flawed information remains in the samples used for training.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the author proposed Mol-LLaMA which could poentially enhance the understanding of molecular features with strong reasoning and explainability. It introduces a novel instruction dataset that includes detailed structural descriptions, structure-to-feature relationship explanations, and comprehensive conversations. The model also integrates complementary information from different molecular encoders, improving the understanding of molecular structures and features. Overall, Mol-LLaMA outperforms existing models in predicting and explaining molecular properties, potentially accelerating scientific discovery in chemistry and biology.\n\nSTRENGTHS AND WEAKNESSES:\n1. Provided chemistry dataset includes detailed structural descriptions, structure-to-feature relationship explanations.\n2. The blending module integrates information from different molecular encoders, improving the accuracy of molecular analysis.\n3. Mol-LLaMA outperforms existing models in predicting and explaining molecular properties, showcasing its potential to accelerate scientific discovery in chemistry and biology.\n\nWeaknesses:\n1. There is a concern about whether LLMs are fundamentally suitable for certain tasks in the benchmark, such as molecular property prediction, which is why only LLM results were presented.\n2. The molecule-to-text evaluation relies exclusively on text similarity metrics (BLEU, ROUGE, METEOR), which are inadequate for assessing chemical factual accuracy.\n3. The evaluation of LLMs based on helpfulness and relevance scores raises doubts about whether these scores accurately reflect the correctness of the information.\n4. The multimodal model and multimodal text data for molecules appear to lack originality.\n\n[1] Liu P, Ren Y, Tao J, et al. Git-mol: A multi-modal large language model for molecular science with graph, image, and text[J]. Computers in biology and medicine, 2024, 171: 108073.\n[2] Liu S, Nie W, Wang C, et al. Multi-modal molecule structure–text model for text-based retrieval and editing[J]. Nature Machine Intelligence, 2023, 5(12): 1447-1457.\n\nQUESTIONS:\n1. Use more nuanced evaluation criteria that consider both text similarity and chemical factual accuracy to provide a more balanced assessment of the model's performance.\n2. Include a wider variety of molecular structures and properties in the instruction dataset to ensure comprehensive coverage like QM9 [1] or GEOM datasets.[2]\n\n[1] Ramakrishnan R, Dral P O, Rupp M, et al. Quantum chemistry structures and properties of 134 kilo molecules[J]. Scientific data, 2014, 1(1): 1-7.\n\n[2] Axelrod S, Gomez-Bombarelli R. GEOM, energy-annotated molecular conformations for property prediction and molecular generation[J]. Scientific Data, 2022, 9(1): 185.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Mol-LlaMA, a multimodal LLM designed for general understanding and reasoning about molecules. The authors create a novel instruction dataset with three types of synthetic data: structural descriptions of functional groups, structure-to-feature relationship explanations, and conversations based on the first two, all generated using GPT-4o.\n\nArchitecture-wise, Mol-LlaMA uses 2D (MoleculeSTM) and 3D (UniMol) molecular encoders to capture structural information. These embeddings are aligned through a blending module, and a Q-Former projects the combined embeddings to the LLM.\n\nDetailed experimental results show that Mol-LlaMA outperforms other baselines across various tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n* The authors present a novel and comprehensive instruction dataset that covers structural, chemical, and biological reasoning data about the molecule.\n* The proposed 2D-3D blending module effectively aligns information from different molecular encoders, and the benefits of this over single-encoder approaches are covered in the experimental results.\n* Qualitative and quantitative experimental results show that Mol-LlaMA outperforms baselines on tasks like property prediction and molecular comprehension.\n\n\n**Weaknesses:**\n\n* The instruction dataset is fully synthetic and heavily relies on GPT-4o. Although GPT-4o is used to filter incorrect data, some hallucinations may still persist. A more rigorous evaluation of the dataset quality would strengthen the work.\n* GPT-4o is used both for generating and evaluating the data, which may introduce bias. Including human evaluations would provide a more robust and independent assessment of Mol-LlaMA’s capabilities.\n\nQUESTIONS:\n* Is there any evaluation performed on GPT-4o's accuracy in extracting functional groups and their connectivity information from the IUPAC notation of the molecule when generating the \"Detailed Structural Description\" data type?\n\n* Please include a full ablation study on other benchmarks like property prediction, MoleculeQA, and 3D-MoIT? This could help to better understand the comprehensive benefits of the Mol-LLaMA architecture and data types.\n\n* Is there any plan to publicly release the data and code?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}