Title: Omni-Mol: Multitask Molecular Model for Any-to-any Modalities
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Models

Molecular Science

Multi-modal Learning
Abstract: In the molecular domain, numerous studies have explored the use of multimodal large language models (LLMs) to construct a general-purpose, multi-task molecular model. However, these efforts are still far from achieving a truly universal molecular model. We identify three key challenges in this endeavor: (1) Existing molecular task datasets are typically small in scale and lack comprehensive domain coverage. (2) Tasks from different molecular subfields are difficult to effectively learn jointly through LLMs due to significant distributional shifts and competition among tasks, which introduces instability in the learning process. (3) Both inter-task and intra-task molecular representations demand different intrinsic dimensions in the language space, making it challenging to balance between redundancy and insufficiency in language model representations. To address these challenges, we innovatively categorize existing small-molecule tasks into four types: Mol2Mol, Mol2Text, Mol2Num, and Text2Mol. We then collect a dataset encompassing over 16 tasks with more than 1.4 million samples, making it the largest molecular instruction-tuning dataset to date. Leveraging the extensive pretraining of LLMs on existing chemical literature, we propose a novel multimodal LLM framework, named **Omni-Mol**, which unifies all small-molecule tasks and supports both molecular generation and understanding. The core of Omni-Mol is our proposed MoGE, which dynamically adapts to the intrinsic rank of different tasks. This mixture-of-experts architecture enhances the model's ability to handle diverse tasks and modalities effectively. Our model achieves unified instruction tuning across 16 tasks and attains state-of-the-art performance on 13 of them. Extensive experiments further demonstrate the scalability and versatility of Omni-Mol.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose  OmniMol, a generalist chemistry LLM with the ability to solve a wide range of tasks. OmniMol is a Llama 3.2 model finetuned with a novel LORA-based approach designed for multitask learning that solves Text2Mol, Mol2Text, Mol2Mol and Mol2Num tasks. Experiments with up to 8B models were done on a multitask dataset with a total of 1.4M samples\n\nSTRENGTHS AND WEAKNESSES:\nThe significance of this research topic is high since generalist chemistry LLMs would benefit the CS as well as the chemistry community well. \n\nThe originality seems medium given that many others have tackled this problem before in a similar fashion. The methodological delta to existing methods is small. E.g., even the early Text+ChemT5 work named as \"pioneering\" by the authors can already do all four mentioned tasks natively (Text+Chem T5 is not explicitly evaluated on Mol2Num but it can solve it naturally without any architectural modification).\n\nOn the positive side, the authors propose a custom LORA-based method (MoGE) which boosts performance. This approach and its usage in a MOE settings is well-motivated and interesting, especially also the ablation study on its value compared to vanilla LORA. \n\nClarity is good overall but can be improved, e.g., the figure ordering is random and they are not referenced in order. \n\nThe results are generally convincingly presented but I have concerns that some of them are cherry-picked (see below)\n\nQUESTIONS:\n- I have some concerns on the results. (1) In some tables OmniMol is incorrectly bolded  (e.g., Molecular Captioning Task, it is not the best). (2) It seems that some cherry-picking on the competing methods was done, e.g., the Text+Chem T5 model achieved better performance on the two tasks (Mol2Text and Text2Mol) of Chebi-20.\n\t- I had a look at the code but it has no README which hampers reproducibility. Also, will OmniMol be available publicly, e.g., via a HF spaces?\n\t- The section on SMILES vs SELFIES gives the wrong perception SMILES may often cause errors in RDKit. I think you want to say that such errors occur frequently *when generating SMILES with a LM*. Please be more explicit here. Also, I think that a small ablation on SMILES vs SELFIES would be useful because only ~55% of the samples have SMILES/SELFIES output whereas ~90% have SMILES/SELFIES input. Then, when you tokenize SELFIES with the BPE-based Lama tokenizer, you may distort the structure of the SELFIES such that not all generated SELFIES are valid anymore. E.g., when you tokenize [C][C] into \"[C\" \"][ \"C\" and \"]\" you arrive at a vocab that can still be used to produce invalid SELFIES like \"][]\". \n\t- P4 L150: Do you mean the hidden size of the LLM's nn.Embedding and the GNN's node embedding size? If really only the sequence lengths differ you just stack horizontally and no padding is necessary (apart from standard padding when you form batches). \n\t- Figure 4: I am buying the scalability argument but the visualization is unfortunate as the bar seems flat -- I would visualize this on a relative scale (y-axis is the delta compared to the first value)\n\nLIMITATIONS:\nThere is a very generic and shallow section about limitations in the appendix.\nThe main paper lacks a substantial discussion of its own limitations", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to build a general-purpose, multi-task molecular model based on a large language model. To this end, the authors mainly focus on and propose the following:\n- A multi-task dataset of 16 molecular tasks collected from public databases by classifying them into four folds: Mol2Mol, Mol2Text, Mol2Num, and Text2Mol. \n- Gradient adaptive LoRA (GAL), which dynamically adjusts the scaling factor with learnable parameters to enhance the multi-task learning.\n- Mixture-of-GAL-Experts, a MoE architecture with the proposed GALs.\n\nThe resulting model, Omni-Mol, is evaluated on the collected multi-task dataset, outperforming the baselines.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\nLeveraging MoE architectures is beneficial for improving performance.\n- In the ablation study, GAL demonstrates performance improvement, indicating its effectiveness in task learning.\n\n**Weaknesses**\n- Omni-Mol is not evaluated on the general tasks beyond the collected dataset. In other words, the proposed model does not exhibit its generalization ability to diverse tasks. Even though the main motivation of this paper is to build a general molecular LLM, the experimental results are limited to showing its effectiveness on its own dataset, not addressing the tackled problem.\n- The proposed method does not guarantee the permutation equivariance of graph-structured data as it directly forwards the node embeddings of graph representations to LLMs. Therefore, if the nodes are permuted, the responses of Omni-Mol will be significantly changed, often generating irrelevant results.\n- As LLMs better understand SMILES than SELFIES, leveraging SELFIES does not fully utilize the internal knowledge in LLMs, which could show inferior performance compared to leveraging SMILES. Even though SELFIES is beneficial to generate valid molecules, using SMILES brings more practicality by fully leveraging the internal knowledge of LLMs and not degrading the original LLMs' abilities.\n- SMILES and SELFIES are not unique, as they could have diverse forms. Specifically, SMILES has diverse forms [1], and as SELFIES is derived from the ordering of SMILES, SELFIES is also not unique. Therefore, using SELFIES could lead misunderstanding of molecular modalities.\n- It is not clear why the multimodal projector is trained only on PubChem. To achieve multi-task learning, training on all the collected datasets sounds more reasonable.\n- The motivation of GLA is not well-established. Motivation experiment for GLA shows that the optimal rank in LoRA could be different according to the tasks. However, it is not directly related to the dynamic scaling factor, as the rank is still static. \n- Related works [2,3,4] are not experimentally compared.\n- The proposed dataset is a collection of well-known instruction datasets, of which the novelty is limited.\n\n\n\n[1] Ganeeva, Veronika, et al. \"Chemical language models have problems with chemistry: A case study on molecule captioning task.\" The Second Tiny Papers Track at ICLR, 2024.\n\n[2] Park, Jinyoung, et al. \"LLaMo: Large Language Model-based Molecular Graph Assistant.\" NeurIPS, 2024.\n\n[3] Zhang, Juzheng, et al. \"UniMoT: Unified molecule-text language model with discrete token representation.\" ArXiv, 2024.\n\n[4] Kim, Dongki, et al. \"Mol-LLaMA: Towards general understanding of molecules in large molecular language model.\" ArXiv, 2025.\n\nQUESTIONS:\n- How important is clipping in GLA? What happens if the clipping is removed?\n- Why are other tasks, such as molecular classification, not considered in this work?\n- Could the authors provide the experimental results on the classification tasks without fine-tuning?\n- What do the left and right y-axes denote in Figure 5 (Middle) and (Right)?\n\nLIMITATIONS:\nThe main limitation of this work is that the proposed model, Omni-Mol, does not show its ability to generalize to tasks beyond the training dataset. I highly recommend evaluating the proposed model on other benchmarks. Additionally, the design of the molecular representations is not reasonable, leaving critical problems such as the permutation equivariance of graph-structured data and diverse forms of string representations.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents Omni-Mol, a model for molecular tasks enabling textual- and molecular inputs and outputs. Omni-Mol uses an LLM-based backbone and further includes a GNN-based graph tokenizer, as well as two methods for handling task-diversity common in molecular learning: GAL is a LoRA variant which can more flexibly learn the optimal rank for different tasks, and MoGE is a MoE approach which allows for different experts to handle different input modalities. Omni-Mol always performs among the best models in a variety of tasks and is ablated to study the effects of different architectural design choices.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- S1: The work considers a very flexible setting which allows one to interleave natural language, with graph- as well as SELFIES-based molecular information.\n- S2: It is an interesting finding that molecular models benefit from MoE to handle multiple input modalities.\n- S3: The topic is very interesting and the move towards flexible multi-modal molecular models is important and timely.\n\n### Weaknesses\n- W1: It does not become clear whether the GNN-based encoder is truly essential for Omni-Mol. This part of the framework is not properly ablated, yet it seems that training Omni-Mol would become much simpler without the additional GNN representation. The paper justifies this design choices poorly. The authors should at least properly justify this design choice in plain text and ideally perform an ablation study to identify whether the GNN is essential for Omni-Mol. Perhaps I am also overlooking that the GNN is architecturally necessary. In that case, the authors should explain this more clearly in the main paper.\n- W2: The experimental results lack error bars or standard deviation. This is despite the fact that the authors claim in the paper checklist that they consider statistical significance. Without standard deviations, the significance arising from the experimental results is questionable. I strongly suggest that the authors repeat the downstream tasks on multiple random seeds to estimate the standard deviation. While the authors do not specify the training time required for one training run of Omni-Mol, I would understand that training multiple models is not feasible within the time of rebuttal. In that case, I recommend to at least run the inference, which inhibits some stochasticity due to the decoder loop of the LLM, multiple times, as described here: https://arxiv.org/abs/2411.00640.\n- W3: In my opinion, the paper lacks a very important baseline: Fine-tuning the same backbone LLM as used in Omni-Mol, using the same data as Omni-Mol, simply treating SELFIES as text. As far as I understand, this baseline is not considered, yet would allow to estimate the actual improvement arising from treating multiple molecular modalities differently.\n\nQUESTIONS:\n- Q1: Can the authors please provide information about the runtime needed to train Omni-Mol?\n- Q2: Can the authors elaborate on the necessity for the GNN encoder?\n- Q3: Could the authors explain whether simply treating SELFIES as text and fine-tuning an LLM on interleaved natural language and SELFIES could be a competitive approach to Omni-Mol, or whether such a comparison was considered?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Omni-Mol, a unified and scalable multimodal large language model tailored for molecular science, capable of handling a wide spectrum of tasks across four modalities: Mol2Mol, Mol2Text, Mol2Num, and Text2Mol. The framework leverages a novel Gradient Adaptive LoRA (GAL) mechanism and a large-scale multitask instruction dataset (>1.3M samples across 16 tasks) to enable effective parameter-efficient fine-tuning. The authors report strong performance on a variety of benchmark tasks, and their model surpasses several larger-scale baselines in terms of both effectiveness and efficiency.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. The authors propose a novel and comprehensive multimodal LLM framework that supports four distinct categories of molecular tasks: Mol2Mol, Mol2Text, Mol2Num, and Text2Mol—this \"any-to-any\" modality design.\n\n2. The authors propose Gradient Adaptive LoRA (GAL), which dynamically adjusts low-rank update magnitudes based on task-specific intrinsic dimensions, thereby enhancing multi-task adaptability.\n\n3. The authors curated a dataset comprising over 1.3 million samples across 16 tasks, making it the largest molecular instruction tuning dataset to date.\n\n## Weaknesses\n1. Despite the claim of “unified instruction tuning,” the work lacks an empirical analysis of robustness to instruction variability. The paper does not report performance under paraphrased or noisy instructions, nor does it assess zero-shot generalization to unseen prompts.\n\n2. While Omni-Mol demonstrates strong results on most tasks, its performance on the Catalyst Prediction task lags behind several baselines. This suggests potential difficulty in modeling subtle structure-function relationships specific to catalysis, which may involve sparse and high-variance distributions.\n\nQUESTIONS:\n1. How sensitive is GAL’s performance to hyperparameter tuning?\n\n2. How well does the model generalize to truly novel molecular tasks not seen in training?\n\n3. In tasks requiring multiple molecules as input (e.g., reactions),  how graph encoder handle multi-input task?\n\nLIMITATIONS:\n1. Due to limited computational resources, the current study is unable to scale Omni-Mol to larger parameter sizes. This restricts the exploration of potential performance gains from model scaling or deeper expert routing.\n\n2. Although instruction tuning is central to the framework, many prompts are manually templated and task-specific. There is no mechanism for dynamic prompt generation or adaptation, potentially limiting model usability by non-expert chemists or in unstructured settings.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}