mCLM: A Modular Chemical Language Model that Generates Functional and Makeable Molecules
Authors: Carl Edwards
Chi Han
Gawon Lee
Thao Nguyen
Sara Szymkuć
Chetan Kumar Prasad
Bowen Jin
Jiawei Han
Ying Diao
Ge Liu
Hao Peng
Bartosz Andrzej Grzybowski
Martin D. Burke
Heng Ji
Venue: ICLR 2026 Oral (2026)
Keywords: molecule-language multimodality
language model
molecule tokenization
molecule generation
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose mCLM: a bilingual, modular Chemical-Language Model that understands both natural language descriptions of functions and molecular blocks; mCLM front-loads synthesizability while improving the functions of molecules in a principled manner.

Abstract: Despite their ability to understand chemical knowledge, large language models (LLMs) remain limited in their capacity to propose novel molecules with desired functions (e.g., drug-like properties). In addition, the molecules that LLMs propose can often be challenging to make, and are almost never compatible with automated synthesis approaches. To better enable the discovery of functional small molecules, LLMs need to learn a new molecular language that is more effective in predicting properties and inherently synced with automated synthesis technology. Current molecule LLMs are limited by representing molecules based on atoms. In this paper, we argue that just like tokenizing texts into meaning-bearing (sub-)word tokens instead of characters, molecules should be tokenized at the level of functional building blocks, i.e., parts of molecules that bring unique functions and serve as effective building blocks for real-world automated laboratory synthesis. This motivates us to propose mCLM, a modular Chemical-Language Model that comprises a bilingual language model that understands both natural language descriptions of functions and molecular blocks. mCLM front-loads synthesizability considerations while improving the predicted functions of molecules in a principled manner. Experiments on 430 FDA-approved drugs showed that mCLM is capable of significantly improving chemical functions critical to determining drug potentials. mCLM, with only 3B parameters, also achieves improvements in synthetic accessibility relative to 7 other leading generative AI methods including GPT-5. When tested on 122 out-of-distribution medicines using only building blocks/tokens that are compatible with automated modular synthesis, mCLM outperforms all baselines in property scores and synthetic accessibility. mCLM can also reason on multiple functions and iteratively self-improve to rescue drug candidates that failed late in clinical trials (“fallen angels”).

Published as a conference paper at ICLR 2026
MCLM: A MODULAR CHEMICAL LANGUAGE MODEL THAT GENERATES FUNCTIONAL AND MAKEABLE MOLECULES
Carl Edwards
Chi Han
Gawon Lee"
Thao Nguyen
Sara Szymkuć
Chetan Kumar 2rasad2
Bowen Jin
Jiawei Han
Ying Diao
Ge Liu
Hao Peng
Bartosz A. Grzybowski
Martin D. Burke
Heng Ji
ABSTRACT
Despite their ability to understand chemical knowledge, large language models (LLMs) remain limited in their capacity to propose novel molecules with desired functions (c.g., drug-like properties). In addition, the molecules that LLMs propose often be challenging to make, and almost never compatible with automated synthesis approaches. To better enable the discovery of functional small molecules, LLMs need to learn new molecular language that is more effective in predicting properties and inherently synced with automated synthesis technology. Current molecule LLMs are limited by representing molecules based atoms. In this pa- per, we argue that just like tokenizing texts into meaning- meaning-bearing ssub-)word tokens instead of characters, molecules should tokenized at the level of functional build- ing blocks, i.e., parts of molecules that bring unique functions and serve as effective building blocks for real-world automated laboratory synthesis. This motivates us to propose mCLM, modular Shemical--Langugee Model that comprises bilingual language model that understands both natural language descriptions of functions and molecular blocks. mCLM front-loads synthesizability considerations while improving the predicted functions of molecules in a principled manner. Experi- ments on 430 FDA-approved drugs showed that mCLM is capable of significantly improving chemical functions critical to determining drug potentials. mCLM, with only 3B parameters, also achieves improvements in synthetic accessibility relative to 7 other leading generative Al methods including GPT-5. When tested on 122 -fi-tribiiiiitiinnnn medicines using only building blocks/tokens that are compatible with automated modular synthesis, mCLM outperforms all baselines in property scores and synthetic accessibility. mCLM can also reason on multiple functions and iteratively self- self-improve to rescue drug candidates that failed late in clinical trials ("fallen angels").
Data and Model
Code
INTRODUCTION
Small molecules-the the class of chemical matter primarily built from carbon atoms bonded together- can perform wide range of important functions in human society (Zhang et al. 2025). These
include essentials like promoting health by acting as medicines (Zheng et al. 2024] Edwards et al. 2024b; 2024b)(Singhan (Singhal et al. 2023) Thirunavukarasu et al. [023) Xiao et al.) 2024a), converting energy by functioning key components in solar cells (Nguyen al.) 2024b Lv et al. 2021 Li et al. 2023b 2024b Si et al. 2024), and achieving sustainability by serving as inherently recyclable products. These functions also include many nice-to-haved that drive substantial economic growth, including colorants, flavorings, perfumes, cosmetics, coatings, quantum dots and insect repellants.
'Equal contribution. 'Siebel School of Computing and Data Science, University of Illinois Urbana- Champaign, "Department Chemistry, University of Illinois Urbana- Champaign, "Allchemy Inc., 'Department of Chemical and Biomolecular Engineering. University of Illinois Urbana- Chaapa-Caampainn *Ulsnn National Institute of Science and Technology, "Institute of Organic Chemistry. Polish Academy of Sciences. Correspondence: one2@i11 1 noiss edu, chihan33i11 incis. edu, hengjiß 8 noi iso.s... edu, moburke@: inniiinsiis incis.edu

---

Published as a conference paper at ICLR 2026
4-IN- Mil-Mettthoieennnneeeeeet
d-IN- Methylapeeeerrl methy!
Natural language
beeoroi and group acts activation loop binder and promotes solublity and orail bs bicactvvity in imatinib.
Beezzn and group wets activation
loog binder and prometes solubiiity and brail bxo activity imatinia
Peptides
MA
Small molecules
DOOO
Synthara Frinddyy-dy Pundion intaad LLM-eradded
Figure I: mCLM adopts a modular chemical vocabulary, which uses synthesis robot- friendly molecu- lar building blocks as tokens together with natural language tokens. Compared with using natural language names, SMILES strings, or holistic embeddings for whole molecules, this building block level of tokenization better enables the prediction of compounds with improved properties, and guarantees automated synthesizability a priori, thus building a direct link between the digital and physical worlds. This approach stands to substantially enable Al-guided discovery of new small molecules with targeted functions.
The traditional approach for small molecule synthesis highly artisanal, and thus slow and expensive: (1) It is unfriendly to automation: machines are not good at performing thousands of reaction types, with each run under thousands of possible conditions and using millions of possible starting materials. (2) It leads to undemocratized landscape in drug discovery. With development costs averaging around $1.3 billion per drug, only economically advanced countries can afford to invest such high- risk research (Kneller) 2010). It also excludes potential breakthroughs from the developing world and inhibits research that could prevent diseases prevalent there. Moreover, participation in the process of molecular innovation currently requires access highly trained experts in chemical synthesis. As result, there are no effective drugs for many diseases such as Parkinson and Amyotrophic Lateral Sclerosis. (3) Most importantly, there are many instances of known commercial drugs or materials that have well-documented limitations that have remained unaddressed. For example, Imatinib is an important anticancer drug that works well in most parts of the body. but it only poorly penetrates the Mood-brain barrier (Takayama et al. 2002 Senior 2003) Coutre et al.) 2004 Isobe et al. 2209)) This means may effectively treat cancer its primary site, yet fail to prevent fatal brain metastasis. In such cases, more blood-brain- brain-bbrrierr penetrant molecules that retain all of the other favorable properties of the current drug are desired, but making such selective modifications in functional properties can be very challenging. As another example in the materials domain, 350+ million people in Asia and the Pacific have only limited access to electricity, and 150 million people still have no access at all. Organic photovoltaic (OPV) molecules are expected to be more economical and environment trrrommmnt-rrridllyyy alternatives to current solar cells. However, current commercialized OPV devices either achieve less than 10% energy conversion siciiency--igniffcaane lower than traditional silicon solar cells-or or have stability far short of 10 years (Solak and Irmak 2023).
An alternative block chemistry approach for small molecule synthesis has recently emerged (Gillis and Burke 2007 Woerly et al. 2014:11 Li et al. 2015aa Lehmann et al. 2018 Trobe and Burke 201 8
al. 2024a). Block chemistry is iterative carbon-carbon bond-forming chemistry that machines can do. It iteratively
Blair et al. [222aa] Angello et al. 2022 Wang et al. 2024| Strieth-Kalthol
assembles small molecules from prefabricated building blocks using chemistry that is simple and general and thus readily automated. Akin to automated DNA, RNA, and peptide synthesis platforms, a major strength of this block-based approach is that it can access billions of novel small molecules with high degrees of functional potential using only few automation-frien friiendly reactions and bounded set of pre-fabricated function-infuse building blocks. We specifically recognized that this block-based approach provided an opportunity to create new modular language for chemistry. The idea that molecules and their synthesis may be best understood from the perspective of a "chemical language" dates back to 2014 (Cadeddu et al., 2014), where structural fragments and functional groups play the roles "chemical words". This view is supported by multiple observations aligned with natural language: (1) they be decomposed and reassembled, (2) they exhibit ambiguity- the
2

---

Published as a conference paper at ICLR 2026
same building block can perform vastly different functions depending on the chemical context, and (3) they possess significant diversity, as many different structures can lead to the same function. This linguistic parallel suggests the potential to train a large language model specifically for molecules by linearizing their structures into modular sequences.
A common representation for such an approach is the Simplified Molecular Input Line Entry System (SMILES: Weininger 1988), where atoms are denoted by one- or two-character symbols (e.g.. C for carbon, Br for bromine, and F for fluorine), rings are represented with numbers, and branches are indicated using parentheses. However, such atom-level tokenization strategies (e.g., SMILES (Weininger 1988) or SELFIES (Krenn et al., 2020 resemble character-level natural language models, which struggle to generalize effectively. Unlike proteins, which have a fixed vocabulary of 20 amino acids, small molecules exhibit an open vocabulary when each atom is treated as a token, as illustrated in Figure T It also causes severe restrictions in practicality because many of the new structures proposed by LLMSs based on atom-level tokenization are not practically synthesizable in the laboratory. This approach has created major gap between what is now possible in silico and what is possible in the physical world.
Furthermore, the SMILES representation can obscure critical structure information: two atoms that are direct neighbors in the molecular graph may be distantly separated in the SMILES string. Even with recent efforts to align SMILES and natural language description (Edwards al. 2022a; Pei et al. 2024| Ahmad et al. 2022). integrate chemical properties and functional groups (Nguyen et al. 2025) into SMILES, incorporate 3D geometric information (Fu et al., 2025 Li et al.; 2025a). and employ graph neural networks to capture molecular graphs and chemical reaction contexts (Wang et al. 2022a), these representations still fail to encapsulate functional knowledge that is often described only in natural language literature because the inherent properties and functions of molecules are hidden in their structure, composition, and interaction.
Therefore, our goal is not to make all drug-like molecules; rather, our goal is to figure out how to make the right ones, better, faster, stronger. Unlike machines, human scientists are inherently "multilingual." seamlessly navigating diverse modalities ffro-ffrommmo natural language and scientific figures in literature to complex scientific data such as molecular structures in knowledge bases. contrast, most prior work on molecule discovery trains large language models on single modality. Moreover, Human scientists "think before they talk," grounding their reasoning in deeply reflective and deliberate reflection and critical evaluation to generate new hypotheses. Current models lack this critical thinking capacity, limiting their ability to contribute meaningfully to discovery. In particular, the human body is a highly complex, interconnected system, and drug discovery is essentially multi-objective optimization problem. It involves balancing factors such as drug absorption, first-pass metabolism, bioavailability. distribution, protein binding, and clearance. However, improving one property often comes the expense of another. Many promising drug candidates have failed in the final stages of FDA approval due to an unacceptably high risk of drug-induced liver injury.
Against this backdrop, we argue that there is a correctable fundamental mismatch between the way LLMs work and the way chemists traditionally synthesize and study small molecules. Reducing tokenization granularity to the level of individual letters is counterproductive, as it complicates meaning extraction and increases the likelihood of generative Al hallucinating words that don't. t exist. Analogous limitations are inherently linked to atom-based tokenization. And as a result, unfortunately, much of the generative Al research for scientific discovery within the computer science community does not extend to hypothesis verification in the physical world. To bridge this gap, in this paper, we propose a novel LLM by drawing inspiration from the scientific discovery process itself. We aim to develop a science- science-inspired large language model that follow three principles: (1) "Observe" " acquire, represent and integrate knowledge from multiple data modalities; (2) "Think" think critically generate hypotheses; and (3) "Propose" verify hypotheses through the Physical World. We aim to teach computers to speak two complementary languages: one that represents molecular building blocks (i.c., subgraph structures) indicative of specific functions and compatible with automated modular assembly, and another that describes these functions in natural language (Figure 2) Unlike existing approaches that add such knowledge as post hoc step, we develop function- and synthesis- aware modular chemical language model (mCLM). Inspired by bilingual speakers who frequently "code-switch" (naturally and often switch between their two languages within the same message: Poplack 2013). we propose novel neural encoder that integrates molecular structure and natural language. mCLM incorporates both function- and synthesis-related knowledge into the small molecule tokenization process a priori. First, we tokenize small molecules at the level
3

---

Published as a conference paper at ICLR 2026
Text Molecule Bilingual Language
Training mCLM with Modular Chemical Reasoning Knowledge
MCLM output
Malecule Module Library
Chemical feature space
Leaminng Molecular Functions With Natural Language sc hale binder
GUN
Similarity search
Ecop bincee
Attraatin loop lanale
Transformer Decoder Layer
promoter
mCLM
Transformer Decoder Layer
Transformer Decoder Layer
linge bandee promidio
Related Chemical Knowledge Funetional groups that promote solubilty: (Menty Piperazine, Benoeng, Carbanyl ...
CWN
CHN
In imatinib [MOL] To
[/MOL] N- M-Methylpiperadnn ring [MOL]
[/MOL]
loop binder promotes solubility bioactivity.
last-step synthesis reaction of imatinib
[MOL]
"2'"
[MMOL]
Motecule Module Library
Figure 2: An overview of the mCLM. It is a multimodal themical-languaal model that tokenizes molecules into synthesis robot-friendly building blocks, thus creating direct link between the digital and physical worlds. After being trained on datasets consisting of properties, functions and synthesis data, the mCLM can conduct critical chemical reasoning through iterative refinement process.
of building blocks (graph substructures) that are able to predict function and are, by design, flexible, multi-scale, fully compatible with automated modular small molecule synthesis. We use graph neural networks to encode each building block. We then extract natural language sentences from the literature that describe the molecule's ' functions and chemical reactions and synthesis constraints of various molecules, and seamlessly insert the encoded building blocks alongside the corresponding entity names to form the training data, as illustrated in Figure 2 By reasoning on such functional building blocks, mCLM guarantecs to generate efficiently synthesizable molecules thanks to recent progress block-based chemistry, while also improving the functions of molecules in a principled manner. During model inference, mCLM enables these functional modules to be predictably and automatically assembled into molecules with desired functions.
Compared to previous work, mCLM has multiple potential advantages of encoding and generating molecules at a modular level:
1. Synthesis efficiency: proposed molecules can be faster and more broadly makeable because the process is, by design, simple, iterative, general, and muchine-friendyy This can enable rapid iterative drug and material development via seamless integration with automated lab experiments.
2. Multimodal understanding: resembling the mechanism of natural language tokenization, molec- ular moduland Iadularizztion provides more natural interface to align with word representations to form more powerful multimodal representation.
Deliberate reasoning: leveraging LLMs' instruction following capability and wide-scale pretrain- ing, mCLM is able to iteratively refine molecules based knowledge of molecular functions.
2 THE MODULAR CHEMICAL LANGUAGE MODEL
In natural language modeling, tokenization identifies common substrings (such words and sub- words), which carry richer semantic information than sequences of characters. Similarly in nature, most small molecules are composed primarily of connected building blocks (Lehmann al. 2018 Trobe and Burke 2018). There is likewise high degree of inherent modularity in many medicines and materials EEtll and Schuhmann 2019 Arkan et al., 2020 Andrews et al.) 1984, Vitaku et al. 2014). To leverage this phenomenon in chemistry and borrow the spirit from natural language modeling, we propose mCLM, a multimodal model that jointly encodes and understands natural language and molecules based synthesis-friende building blocks instead of atoms. In Section 2.1
4

---

Published as a conference paper at ICLR 2026
introduce the concept of molecular building blocks chemical "vocabulary" and describe the tokenization process to obtain the library of building blocks. Then in Section [.2 we describe the mCLM architecture and its training. Finally, we introduce the reasoning mechanism of mCLM which refines molecule design over multiple iterations in Section 2.3
2.1 A DUNCTION-INFUSER AND SYNTHESIS SONTHESIS-FREENDYY VOCABULARY
In this work, we propose leverage chemical vocabulary V of synthesis-frient building blocks. This approach guarantees capacity for automated iterative assembly first reported in 2015 (Li et al. 2015b) and, since then, demonstrated in multiple experimental campaigns including Strieth-Kalthor et al. (202bb) and Angello et al. (2024). The blocks can be chemically connected using predefined synthesis rules in short period. Briefly, akin to language models developed for peptides/proteins, small molecules can be assembled automatically from makeable building blocks. These building blocks are often highly associated with chemical functions, such as binding to protein targets, modulating enzyme activity, or affecting involved metabolic processes. This will enable rapid and iterative proposal of new small molecules, automated synthesis of those small molecules, and generation of the corresponding functional data demand. In contrast to SMILES strings which break molecule structure during graph linearization (c.g., separating physically adjacent subgraphs such the two carhons in molecule C(N)C), our representation is linear in the physical world.
Sunuki-Myyuur Coupling
SymbhesioGuurartteed Guaranteed Tokenizer
Coverage Check
Rule-based Tokemizer
Toke-isstion niles constroints guo onttc synthesitabiity
Check structural torget using generated tokens
Takenization rules only ensures hull coverage
Buchwrald Harbwig Coupling
Amido Goupling
on
Pass
Tokenization Rules
Panttional
Used for protraining
Figure 3: An overview of the tokenization process. A functional molecule is first processed by the monthesis-guaranten tokenizer to produce a set building blocks compatible with automated modular synthesis. These blocks are then evaluated via a structure coverage check to determine whether they fully reconstruct the original molecule. If coverage is complete, the blocks are used directly for pretraining. Otherwise, the molecule is reprocessed using rule-based tokenizer to ensure full representation for training purposes.
During dataset construction, combine two tokenization strategies to balance coverage and synthetic feasibility: a synthesis-uuaranteed guaranteed tokenizer disconnects the molecule only bonds that can be formed by predetermined small set of reactions that can be performed in an automated manner. Specifically, it only permits bond disconnections that correspond reactions compatible with state-of-the-ar automated synthesis platforms. These three bonds are: amide coupling. Suzuki-Miyaura coupling, and Buchwald-Hardy coupling (see tooknnizatoiiiiioonnn rules: Tyrikos-EEgass et al. (025).. The disconnection rules are adapted from well-established principles in black-based and automated chemistry (Blair et al.) 2022b Li et af, 20155). Qualitative examples of tokenization are included in the appendix [F] and the full rules and tokenizer code will be released with documentation. Importantly, the tokenizer is modular: as new automated reactions are validated in practice, their corresponding disconnection rules can be readily incorporated, expanding the makeable chemical space without requiring fundamental architectural changes. Unlike traditional retrosynthesis tools that prioritize maximum coverage or human-designed heuristics, this tokenizer is guided by the operational constraints of block chemistry. also ensures that resulting blocks are free from functional group conflicts that would interfere with downstream synthesis. This conservative approach leads to set of molecular tokens that
guaranteed tokenizer and a rule-based tokenizer (Figure 33.. The synthesis-
5

---

Published as a conference paper at ICLR 2026
guaranteed to work under defined synthesis protocols, enabling seamless transition from model- generated output to real-world synthesis. When the synthesis guaranteed tokenizer cannot fully cover reaction constraints fall back rule-based
syathesis-guaranteed
molecule- due to chemical incompatibilities tokenizer to ensure coverage of a larger variety of molecules. The rule-based tokenizer also breaks molecules along the same automated machine-friendly bonds the above tokenizer. However, no other rules are applied beyond specifying a minimum size for blocks. This rule-based tokenizer used only during training to support learning on diverse molecular structures while maintaining consistency with synthesis logic. We note that the vocabulary growth is analogous to the shift from character- level to subword-level tokenization in natural language, where larger but semantically meaningful tokens significantly improved performance. These reusable, synthesis-friendd building blocks are derived from real data distributions. We collected approximately 8 million molecules from a wide range of public sources (Appendix G of which approximately 6.8 million were tokenizable using our full tokenization pipeline (see Figure This process resulted in a vocabulary of 800,000 unique building blocks with 200,000 synthesis-guarand guaranteed (see Table 6 An example of how the block sequence can reconstruct the molecule structure is also provided in Appendix
2.2 CHEMICAL-LANGUAA MODELING
Figure 2 fillustrates the architecture of mCLM. The model a sequential generative model that processes molecule and text sequences in a unified manner. It adopts Transformer architecture, which is well-suited for handling sequential data and allows for using pre-trained language models as a backbone. After tokenizing the molecules as mentioned in the last section, we encode each building block using graph neural networks (GNNs; Edwards et al. 2024a Sprueill et al. 2024 Gasteiger al.) 2021) before passing them through an adaptor module. These representations
then concatenated with natural language embeddings at positions where molecule entity names appear. This results in form of "'code-wwithhr
language which blends molecular structures with natural language descriptions. The feature sequence is then fed into a Transformer decoder-only architecture, which predicts the next token based on previous tokens. This allows for pre-training the model on a large corpus of multi-modal data, enabling it to learn and "talk about" the relationships between the different modalities and their respective representations.
We train mCLM on top of open-source pretrained large language models (we adopt Qwen2.5 Yang al. (2024) as the starting point), allowing us to leverage their natural language understanding and reasoning capabilities without incurring the computational cost of training from scratch. Furthermore, pretrained laaural- L.LM have been exposed to broad range of scientific information, making them a strong foundation for our curriculum. They help the model acquire general scientific knowledge before progressing to domain-speciff training on molecular structures and drug-related information. As the training objective, we adopt unified categorical cross-entropy (CCE) loss applied to both natural language and molecular tokens:
(c,To. ev, U.E E Viattural language
C= (o()), logit (Po(u) xx-.a-1))
[c fo (c)N
Vmolocular building block
Specifically, the cross entropy loss is computed between the ground truth distribution P(x) and the model distribution (x) over the combined vocabulary of natural language and molecular building blocks. The model generates logits for the next token v by computing the dot product between the contextual representation c; with token embeddings. C; is produced by the Transformer given 1)). For natural language tokens, the embedding e, is directly taken from the pretrained natural language model. The embedding for molecular building blocks is computed by passing the building block's graph through a GNN, followed by a linear adapter function for to project it into the same embedding space. This formulation enables joint training over both modalities using a single loss function. We train on approximately 1 million examples consisting of the 1,000 most frequent molecular building blocks, sampled from a new dataset we constructed. This dataset contains over 36 million molecular instructions, 1, 1,000 tasks, and million molecules. More details on the training procedure are described in Appendix D Notably, the architecture of mCLM is compatible with larger vocabularies, and we anticipate future scaling studies as computational resources permit. mCLM is able to generalize to molecules composed of unseen building blocks. During training, the model learns associate chemical structure with function via instruction-based pretraining. During inference, new blocks can be directly incorporated without retraining.
the previous tokens X X [X

---

Published as a conference paper at ICLR 2026
2.3 CRITICAL CHEMICAL REASONING
Chemical reasoning over molecules often involves optimizing multiple functions, such as toxicity, bioactivity, and binding affinity. Therefore, molecule structure, especially with only single attempt. Optimizing one function may lead to trade-offs in others. For example, many drugs with higher potency were rejected due to increased toxicity to patients. To address this, we propose a reasoning process that allows the model to refine its own generated molecules and iteratively improve their desired functions. At each iteration, we evaluate the properties and identify one that still requires improvement, and mCLM proposes modification the molecule targeting this property. This process is repeated until a maximum number of iterations is reached. This process is summarized in Appendix Algorithm
not a straightforward task to propose an ideal
3
EXPERIMENTAL EVALUATION
3.1 CREATING ORACLE MODELS FOR EVALUATION
To evaluate the performance of our generated molecules, we construct oracle models focused on Absorption, Distribution, Metabolism, Excretion, Toxicity (ADMET) property prediction. We select 6 tasks from the Therapeutics Data Commons (TDC) benchmark (Huang et al. 202T]: AMES (mutagenicity), BBBP (blood-brain barrier permeability), CYP3A4 inhibition (metabolism). DILI (drug-induced liver injury), HIA (human intestinal absorption), and PGP (P-glycoprotein substrate classification). The detailed training procedure described in Appendix D.2.3
3.2 IMPROVING FDA-APPROVED DRUGS WITH OUT-OF- VOCABULARY BLOCKS
In practice, real-world drug discovery is often driven by the optimization of known molecules, as existing approved drugs offer more direct and promising path safe and effective new drugs. Therefore, we evaluate mCLM's drug proposal capability in improving FDA-approved drugs. Specif- ically, we apply the mCLM to improve the properties for all FDA-approved drugs consisting of synthesis-guarante confine the output vocabulary of mCLM to 582 (120/122) contain blocks that were not present in the 1,000-block training vocabulary, presenting opportunity to examine how the mCLM works on out-of-distribution molecules. Results in Table show that improvements are achieved for all propertie This shows that mCLM is not limited to a fixed vocabulary but can generalize beyond its training coverage. We also compare our approach against wide range of strong text-basee bastd molecule editing baselines, including MoleculeSTM (Liu) (Gemini-2.5-F)) LDMol (Cang and Yel 2024), and Claude 3.5 Haiku (Claude-3.5-HH) We conducted additional comparisons with two relevant baselines: DGAE (Boget et al. 2024) (a discrete graph autoencoder using vector quantization) and HierVAE (Jin al.) 2020)) (a hierarchical VAE for molecular graph generation). The detailed results and technical comparison with pettrr-uanttzed methods are enclosed in Appendix [H] Despite the fact that all of the baseline models are allowed to generate without synthesis guarantees, their average improvements fall behind mCLM.
blocks. This amounts to 122 molecular structures and 153 unseen blocks. We gubbbsss-g blocks. Most of these drugs
et al., 2022), FineMolTex ([.i et al., 20255). GPT-4o, GPT-5, Gemini-2.5-Fles
Table I: Average pharmacokinetic and toxicity properties of FDA drugs composed of synthesis- guaranteed blocks, as well as their proposed modifications. (J: lower is better, 1: higher is better). Green better than FDA, Red worse, Light green bold best overall per column.
Model
AMES (L)
BBBP (1)
CYP344 (4)
DILI (L)
HIA (t)
PGP (4)
Avg. Improv.
FDA Drug
47.8
614
60.1
98.96
64.6
0.00
MoleculeSTM
47.1
63.4
22
98.76
64.1
0.31
FincMolTex
47.5
66.0
2.4
59.5
98.84
64.0
-0.73 %
LDMol
49.0
63.5
2.5
57.5
99.0
67.6
-3.07 %
GPT-40
46.0
2.2
99.3
2.45
GPT-5
49.1
70.2
2.4
61.2
99.2
65.0
-0.82 %
Claude-3,5-Haiko
49.3
72.2
2.2
58.6
98.3
65.7
1.64
Gemini-2.5-Flash
45.0
72.5
1.8
58.1
99.1
64.5
6.98
mCLM (Ours)
44.4
85.2
1.4
53.7
98.99
64.4
15.0
The key to expediting the drug creation process is to discover potent molecular candidates that are simultaneously synthesis-friendly. While mCLM shows strong property editing results, its key benefit
Wee also test different distribution of 430 non-symtherix-guuraateedd FDA drugs in Appendix A.2
7

---

Published as a conference paper at ICLR 2026
lies in its puathesis-friendle nature. We assessed the synthesizability of generated molecules by computing synthetic accessibility (SA scores) (Ertl and Schuffenhaueri (2009) as a quick heuristic (see Table B). Then, more rigorous assessment, of-the-art retrosynthesis software (Wotos et al. 2022 Strieth-Kalthol et al. 2024c). Allchemy is computationally expensive, but it evaluates synthesizability the best of publicly available human chemical knowledge. For example, it finds synthetic routes for 98.1 % of the FDA-approved molecules. Moreover, to our knowledge is the only retrosynthesis software for which many of the proposed routes have been reduced to practice in the lab with physical experimentation. These studies have been published in top tier journals (Mikulak Klueznik al., 2020). We select the top natural language models by SA score (mCLM, MoISTM, and FineMolTex, and randomly sample 200 generated molecules from each to be assessed by Allchemy on a supercomputing cluster.
consider Allchemy which is the state-
Table 2: Synthetic accessibility (SA) (Ertl et al. 2009). validity, and retrosynthetic results across baselines. Synthesizability is the percent of valid molecules where retrosynthetic route was found. Makeability is the overall percent generations which can be synthesized (Makeability =Valid x Synth.).
Model
SA (4)
Validity (%)
Synthesizability (%)
Makcability (%)
FDA
2.70
100.0
98.11
98.11
MoleculeSTM
2.64
93.80
91.03
85,39
FineMolTex
2.58
94.20
90.15
84.96
mCLM (Ours)
2.43
100.0
98.23
98.23
mCLM (No GNN)
2.97
45.21
94.66
42.8
mCLM (No Synth. Tokenizer)
3.09
100.0
73.65
73.65
As shown in Table 2 molecules proposed by mCLM 100% valid (syntactically correct) and 98.2% synthesizable, which is superior even to the FDA drugs. In contrast, MoleculeSTM outputs valid only 93.9% of the time, and among those, only 90.3% are predicted be synthesizable using exhaustive retrosynthesis search. Out of MoleculeSTTMgeeneatte molecules, only 84.8% can be made (93.9 x 0.903 84.8).
3.3 CASE STUDY: MULTI-STEP REASONING TO RESURRECT THE "FALLEN ANGELS"
There are many new drug candidates that almost reach FDA approval but fall short for various reasons when tested in clinical trials. For example, Evobrutinib is Bruton's 's tyrosine kinase (BTK) inhibitor that went through clinical trial as a drug for relapsing Multiple Sclerosis. However, the FDA placed partial clinical hold on Phase III trials in April 2023 after two patients showed signs of drug-induced liver injury (Montalban et al., 2024). TNG348 is a USP1 inhibitor designed for treating BRCA1/2-mutant and HRD cancers, but it failed in phase 1/2 clinical trials due liver abnormalities Inc 2024 Simoneau et al., 2025). These "fallen angels" represent tremendous opportunities for impactful engagement of the Al/chemistry interface, because much is known about the strengths of each of these small molecules, and it is also known why they fell short. Fixing such fallen angels is high leverage opportunity for the function-infused mCLM to contribute.
Figure 4 shows an application of the mCLM, without synthesis restrictions on the vocabulary, to these two molecules. For both, the initial step to optimize DII.I, the reason the drugs failed in clinical trials. Following that, the mCLM fixes other properties which were made worse in the previous attempt (PGP for Evobrutinib and BBBP for TNG248). For good measure, another property of each molecule is then improved. Notably, at each step, the mCLM only makes minor modifications to each drug of roughly building block. While the mCLM shows promising results for repairing these drugs, it is worth noting that drug discovery is mbny-objective optimization problem. While we able to generate molecules with improved toxicity relative to Evobrutinib and TNG348, as well as other properties, yet other important properties may still have been compromised. Future work may want to investigate longer reasoning chains across a wider variety of properties. For comparison, we show (less-successful) version of this experiment using MoleculeSTM (Appendix A.3)
8

---

Published as a conference paper at ICLR 2026
Decrease DILI
Decrease PGP
Decrease AMES
Drug name:
Evobrutinib
Target: BTK
Reason for
DIL:: 82.7
DILI: 50.0
DILI:
DILK 75.4
faiture:
PGP: 59.3
PGP: 67.6
PGP: 59.2
PGP: 58.4
Liver toxicity
AMES: 77.2
AMES: 40.2
AMES: 63.6
AMES: 52.2
Drug name:
TNG348
Target:
USP1 enzyme
Reason
DILL:
84.8
DILI:
73.3
DILE
69.3
DILI:
64.7
failure:
BBBP:
100
BBBP:
2.31
BBBP:
100
BBBP:
100
Liver toxicity
CYP3a4: 0.35
CYP3a4: 2.48
CYP3a4: 2.06
CYP3a4: 1.30
Decrease DILI
Increase BBBP
Decrease CYP3a4
Figure 4: Examples of fallen angel property modification.
3.4 LIBRARY DESIGN: IDENTIFYING FUNCTIONALLY- BLOCKS
As a by-product, mCLM also able to identify blocks which are preferred for certain functions. This can help inform virtual screening campaign design (e.g., which blocks should use to get the best chemical space for BBBP?), and can also be useful for stimulating scientific inquiry. As example, modifications preferred by the mCLM to improve DILI in FDA drugs (Figure D). Notably, modification 3 resembles newly discovered modification for reducing amphotericin toxicity by human scientists in Maji al. (2023)
calculated the most frequent
MCLM
HD
OH
RELATED WORK
Protein language models (Lin et al.) 2023 Hayes et al.
2025 Wang et al. 2025b Madani et al. 2023 Xiao al.,
2024b Su al. [2024] Buehler and Buehler 2024 Wang
al. 2025al Zhu et al. 2024 Ferruz al.) 20222 have
been highly successful because proteins/peptides are tok- enized at the level of amino acid building blocks (which are akin to the words a sentence). This makes it possible associate the sequences of these building blocks with the fold and function of proteins, and then the corresponding building blocks can be assembled into targeted sequences.
Four of the most frequent mCLM-proposed modifications to im- prove DILI in FDA-approved drugs.
Figure
Inspired from the successes of protein LLMs, LLMs have been increasingly applied in computational chemistry (Zhang al. 2024a bi Fang et al.] 2023 Livne et al.] 2024] Pei al. 2223 Zhao
et al., 2023/ Yu et al. 2024 Ye et al. 2025 Liu et al. 2023b; Li et al. [024c]] Liu et al.
2024a Taylor et al. 2022). Recent work has further explored cross mooosl modeling of molecule
and language (Edwards al. 2022b Liu et al. 2022 Li et al. 2025b Chang and Ye 2024 Liu
ct al. 2024c). General-purpose LLMs such as GPT-4o OpenAl 2025a), GPT-5 OpenAll 2025b). Gemini-2.5-Flash (Google Deep 2025). and Claude-3.5-H (Anthropic) 2025) show multimodal reasoning capabilities. However, most molecular language models still rely on atom-level (Wang ett al., 2019; Ahmad al., 2022] Zhou et al.] 2023] Xia et al. 2023) Byte- Syir-Pair Encoding SMILES (Chang and 2024), and thus they often produce invalid tokens, misalign with chemical structures, and generate unsynthesizable molecules. To address these limitations, other approaches

---

Published as a conference paper at ICLR 2026
incorporate functional groups and fragments into molecular representations (Li et al. 2023a Nguyen ctt al. 2024c Han et al. 2023 Jin et al. 2024 Nguyen et al. 2024a Wang et al. 2023; Zhang et al. 2020 2021), capturing both atomic- and group-level information. However, these methods tend to break molecules at bonds that are difficult or even impossible to form through chemical reactions. In contrast, our mCLM decomposes small molecules into function-infused and monthesss-frienl] building blocks and integrates synthesis constraints early in training, akin to the way peptide/protein LLMs tokenize at the amino acid level.
Several recent papers propose chemically meaningful tokenization schemes, SAFE NNutahi et al. 2024) modifies SMILES representations avoid fragment splits but still inherits core issues of line notations as we described in Section [ including validity and synthesis challenges. Group- SELFIES [Cheng al. 2023) extends SELFIES with group-level tokens and can in principle represent our synthesis-friennly blocks. If it is used in a line notation form, however, standard LLM tokenizers will not ensure validity or synthesizability. Reasyn (Lee et al. 2025) contemporary to our work, focuses synthesizable molecule projection using reinforcement learning, requiring the model to predict reactions between pairs of blocks. Unlike Reasyn, mCLM does not require the prediction of reactions, since one is guaranteed to exist between any blocks. Their task and training methodology may be relevant for future mCLM models. We also note recent efforts in learning discrete molecule representations with VQ-like mechanisms (Guo et al. 2025 Ha et al.) 2025 (Boget al. (024)). However, these approaches do not guarantee synthesis compatibility and generally lack alignment with natural language. In Section we include comparisons with DGAE and HierVAE additional baselines.
5 CONCLUSIONS AND FUTURE WORK
We have developed a function- and synthesis-aware modular Sntemical-Languagee Model (mCLM), as the first attempt to jointly model natural language sequences with modular chemical language. By design, the mCLM only generates chemical building blocks that can be iteratively assembled on robotic small molecule synthesis platforms, enabling the rapid creation of novel molecules with desired functions, all accessible by speciiiiisss In the future we aim to scale mCLM to larger backbones, incorporate richer multimodal knowledge related to physical and chemical properties from 2D/3D molecular structures, protein-ligand complexes, cell lines and nucleic acid sequences to further enable the mCLM to reason biological activity, protein docking knowledge, and individuals' genetic profiles. We plan to extend chemical reasoning to additional aspects such as filling in unspoken knowledge gaps, thinking outside of the box, System 2 thinking for counterfactual reasoning and plausibility prediction, and resolving conflicting claims. We also plan to leverage more physical constraints from simulation tools and chemical and reaction knowledge bases. In the long term, we envision the mCLM as part of a comprehensive, multi-agent, human-in-the-loop autonomous laboratory, structured around iterative cycles of reasoning, proposal, synthesis, physical testing, feedback, and reasoning to enable never-ending self- celf-improvement and co-evolvement with human scientists.
REPRODUCIBILITY STATEMENT
In section 3.1 and D.2.3 we describe the process and resources of developing oracle models for evaluation. The and Dexpllan the data collection process and the statistics of the training dataset. Section G lists all source datasets. Model training details are described in Section E Evaluation data from FDA- approved drugs are described Section 3.2 which also lists the baselines we used for comparison. The full data and resources will be released upon publication.
gurerstteg tokenizer is described in further detail in Section F Sections C
ACKNOWLEDGMENTS
We would like to thank Kyunghyun Cho, Anna Hart, Hyeonjeong Ha, Gabriele Scalia, Yanru Qu, and Jeonghwan Kim for helpful discussion. This work is based upon work supported by U.S. DARPA ECOLE Program No. #HR001 12390060, and NSF Molecule Maker Lab Institute, an AI Institute for Molecular Discovery, Synthesis Strategy, and Manufacturing funded by the U.S. National Science Foundation under Awards No. 2019897 and 2505932, and NSF NAIRR award. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the U.S. Government. The U.S. Government is authorized to
10

---

Published as a conference paper at ICLR 2026
reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein. This research used the Delta and DeltaAl advanced computing and data resources, which supported by the National Science Foundation (award OAC 2320345 and award OAC 2005572) and the State of Illinois. Delta and DeltaAl are joint efforts of the University of Illinois Urbana- Champaign and its National Center for Supercomputing Applications. We would like acknowledge NVIDIA Corporation's 's contributions to this work through grant of NVIDIA A A100 Tensor Core GPUs.

---

Published as a conference paper at ICLR 2026
REFERENCES
Walid Ahmad. Elana Simon. Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar, Chemberta-2: Towards chemical foundation models. arXiv preprint arXiv:2209.01711 2022.
Allchemy. wtt.httts///lalleee net/
PR Andrews, DJ Craik, and JL Martin. Functional group contributions to drug-receptor interactions. Journal of medicinal chemistry, 27(12):1648 1657, 1984.
Nicholas H. Angello, Vandana Rathore, Wiktor Beker, Agnieszka Wolos, Edward Jira, Rafat Roszak, Tony Wu, Charles M. Schroeder, Alán Aspuru-Guzik, Bartosz A. Grzybowski, and et al. Closed-loop optimization of general reaction conditions for heteroaryl suzuki-miyaura coupling. Science, 378(6618):399-405 Oct 2022. doi: 10.1 00.1166/ciencee.ade8888
Nicholas H Angello, David M Friday, Changhyun Hwang, Seungjoo Yi, Austin H Cheng. Tiara C Torree--Flon Flores, Edward R Jira, Wesley Wang. Alán Aspuru-Guzik, Martin D Burke, al. Closed-loop transfer enables artificial intelligence to yield chemical knowledge. Nature, 633(8029)3351-333,, 2024.
Anthropic. Claude-3.5-H large language model. https://www.anthroo/ 2025-09-22.
2025. Accessed:
Emre Arkan, Eyup Yalcin, Muhittin Unal, M Zeliha Yigit Arkan, Mustafa Can, Cem Tozlu, and Serafettin Demic. Effect of functional groups of self assembled monolayer molecules on the performance of inverted perovskite solar cell. Materials Chemistry and Physics, 254; 123435, 2020.
Dávid Bajusz, Anita Rácz, and Károly Héberger. Why is tanimoto index an appropriate choice for fingerprint- based similarity calculations? Journal of cheminformatics, 7:1-13, 2015.
Guy W Bemis and Mark A Murcko, The properties of known drugs. 1. molecular frameworks. Journal of medicinal chemistry, 39( (0(15)28877225 -893,, 1996.
Daniel J. Blair, Sriyankari Chitti, Melanic Trobe David M. Kostyra, Hannah M. Halcy, Richard L.. Hansen, Steve G. Ballmer, Toby J. Woods, Wesley Wang, Vikram Mubayi, and et al. Automated iterative csp3-c bond formation. Nature, 604(7904):92-97 Feb 2022a. doi:
Daniel J Blair, Sriyankari Chitti, Melanie Trobe, David M Kostyra, Hannah MS Haley, Richard L Hansen, Steve G Ballmer, Toby J Woods, Wesley Wang, Vikram Mubayi, et al. Automated iterative sp 3-c bond formation. Nature, 604(7904):2-997, 2022b.
Yoann Boget, Magda Gregorova, and Alexandros Kalousis. Discrete graph auto-encoder. Transactions on Machine Learning Research, 2024.
Fabio Broccatelli, Richard Trager, Michael Reutlinger, George Karypis, and Mufei Li. Benchmarking accuracy and generalizability of four graph neural networks using large in vitro adme datasets from different chemical spaces. Molecular Informatics, 4108):210032,, 2022.
Eric I. Buehler and Markus J Buchler. X-lora: Mixture of low-rank adapter experts, flexible framework for large language models with applications protein mechanics and molecular design. APL. Machine Learning, 2(2), 2024.
Andrea Cadeddu, Elizabeth K. Wylie. Janusz Jurczak, Matthew Wampler-Doty. and Bartosz A. Grzybowski. Organic chemistry language and the implications of chemical linguistics for structural and retrosynthetic analyses. Angewandte Chemie International Edition, 53(31):8108 8112, 2014. doi: https://doi.orr//I
1002/anie. 201403708. URUlBtpps://onnnnlllbbee wit.....wi.. wiloy, com/ http///o////ss/eee /00 1002/anic 220103700
Jinho Chang and Jong Chul Ye. Ldmol: A text-to-molecule diffusion model with structurally informative latent space surpasses models. arXiv preprint arXiv:2405.17829 2024.
Austin H Cheng, Andy Cai, Santiago Miret, Gustavo Malkomes, Mariano Phielipp. and Alán Aspuru-Guzik
Group selfies: robust fragment-based molecular string representation. Digital Discovery, 2023.
Philipp le Coutre, Karl-Anton Kreuzer, Stefan Pursche, Malte von Bonin, Leopold, Gökben Baskaynak, Bernd Dörken, Gerhard Ehninger, Oliver G. Ottmann, Andreas Jenke, Martin Bornhäuser, and Eberhard Schleyer. Pharmacokineties and cellular uptake of imatinib and its main metabolite cgp74588. Cancer Chemotherapy and Pharmacology, 2004. doi: 10. 1010007/002200033-07741111111110
12

---

Published as a conference paper at ICLR 2026
Carl Edwards, ChengXiang Zhai, and Heng Ji. Text2Mol: Cross-modal molecule retrieval with natural language queries. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 595 607, Online and Punta Cana, Dominican Republie, 2021. Association for Computational Linguistics. doi: 10. 12118663/1/2221.cmm main.47. URL BRLLhttps:// httpp:::/aclaaahhoooooo/// aclanthology 002/2021. cwnlp-malle 447
Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. In Proc. The 2022 Conference on Empirical Methods in Natural Language Processing (2M22,2022), 2022a.
Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pages 375 413, Abu Dhabi, United Arab Emirates, 2022b. Association for Computational Linguistics. URL httpp////aalaanntooooooo.ooo
httplaic...uu..t.aeee//aaaaatoo. wwt.///2...22..ooooo 2022. emn 1aannnp-ttt 1et.ee2.nnmppp-mii22222
Carl Edwards, Ziqing Lu, Ehsan Hajiramezanali, Tommaso Biancalani, Heng Ji, and Gabriele Scalia, Molcap- arena: A comprehensive captioning benchmark on language-enhanced molecular property prediction. arXiv preprint arXiv:241 1.00777, 2024a.
Carl Edwards, Aakanksha Naik, Tushar Khot, Martin Burke, Heng Ji, and Tom Hope, Synergpt: In-context learning for personalized drug synergy prediction and drug design. In Proc. 1st Conference on I Language Modeling (COLMM024), 2024b.
Carl Edwards, Qingyun Wang. Lawrence Zhao, and Heng Ji. L+M-24: Building dataset for Lan- guage+Molecules @ ACL 2024. In Carl Edwards, Qingyun Wang. Manling Li, Lawrence Zhao, Tom Hope, and Heng Ji, editors, Proceedings of the Ist Workshop on Language + Molecules (L+M 2024), pages I- 9, Bangkok, Thailand, 2024c. Association for Computational Linguistics. doi: 10. 1653 URL httt:ttts:: //ac1 antho ogy. wog..org//...o..................
org/ / 2024. 2 langmol-1.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics, 1(1):8, 2009.
Peter Ert and Tim Schuhmann. A systematic cheminformatics analysis of functional groups occurring in natural products. Journal of natural products. 82(5):1258 12t.-1253.-336----- 2019.
Peter Ertl et al. Estimation of synthetic accessibility score of drug-like fikg-like molecules based molecular complexity and fragment contributions. Journal of cheminformatics, 1(1):8, 2009.
Yin Fang. Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang. Zhuo Chen, Xiaohui Fan, and Huajun Chen. Mol-instructions: large-scale biomolecular instruction dataset for large language models. ArXiv preprint, abs/2306.08011. 2023. URL / www.i.wrw...w.............oooort abs/ abs//30..... 22///333...000 09013
Noclia Ferruz, Steffen Schmidt, and Birte Hoocker. Protgpt2 is a deep unsupervised language model for protein design. Nature communications, 13(1 (3(44)4348, 2022.
Cong Fu, Xiner Li, Blake Olson, Heng Ji, and Shuiwang Ji. Fragment and geometry aware tokenization of molecules for structure-based drug design using language models. In Proc. The Thirteenth International Conference on Learning Representations (ICLR2025), 2025.
Johannes Gasteiger. Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural
networks for molecules. Advances in Neural Information Processing Systems, 34:6790-6800 2021.
Anna Gaulton, Louisa J Bellis, A Patricia Bento, Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, Bissan Al-Lazikani, al. Chembl: large-scale bioactivity database for drug discovery. Nucleic acids research, 40(D1):D1 100-D1 2012.
Eric P Gillis and Martin D Burke. A simple and modular strategy for small molecule synthesis: Iterative suzuki-mae miyaura coupling of b-protected haloboronic acid building blocks. Journal of the American Chemical Society., 129(21), 2007. ISSN 0002-7863.
Google DeepMind. Gemini-2.5-4 large language model. htttps://www.deepr..aa com/ 2025. Accessed: 2025-09-22.
Shuhan Guo, Yatao Bian, Ruibing Wang, Nan Yin, Zhen Wang, and Quanming Yao. Unified molecule-text language model with discrete token representation. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, pages 9205-9213, 2025.
Sumin Ha, Jun Hyeong Kim, Yinhua Piao, and Sun Kim. My-clam: Multi-view molecular interpretation with cross-modal projection via language model. arXiv preprint arXiv:2503.04780, 2025.
13

---

Published as a conference paper at ICLR 2026
Shen Han, Haitao Fu, Yuyang Wu, Ganglan Zhao, Zhenyu Song, Feng Huang, Zhongfei Zhang, Shichao Liu, and Wen Zhang. Himgnn: novel hicrarchical molecular graph representation learning framework for property prediction. Briefings in Bioinformatics, 24(5):bbad305, 2023.
Thomas Hayes, Roshan Rao, Halil Akin, Nicholas Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with a language model. Science, 387(6736):850-888 858, 2025.
Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026-1034, 2015.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.0966 006.0985, 2021.
Kexin Huang. Tianfan Fu, Wenhao Gao. Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, Marinka Zitmik. Therapeuties data commons: Machine learning datasets and tasks for drug discovery and development. arXiv preprint arXiv:2102.09544 2021.
Tango Therapeutics Inc. Tango therapeutics announces discontinuation Ing348 program.
May 2024. URL https::/ir..aggotxccom news wtw.c///eeeesseeeeeeeeeeeeeeeeee reeeeeeessss/esssseeesss-s----
tango- thwraphutrrpccssss hww.pppeeettsssanoooneeeeeeee-ee
announces discont wtw.:ttiiiiiiiitttnnnnteeeee prrograa
Y. Isobe, K. Sugimoto, A. Masuda, Y. Hamano, and K. Oshimi Central nervous system is sanctuary site for chronic myclogenous leukaemia treated with imatinib mesylate. Internal medicine journal, 2009. doi: 14.51//99.45-9944444 20090.000990194...
Bowen Jin, Gang Liu, Chi Han, Meng Jiang. Heng Ji, and Jiawei Han. Large language models graphs: comprehensive survey. IEEE Transactions on Knowledge and Data Engineering, 2024.
Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 4839-4848. PMLR, 2020. URL/ttt::/pproccddi: ngg mll wwwpsppsssssp.p.ppp..op v119//in2Ca..htt
Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia Hc, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, al. Pubchem 2023 update. Nucleic Acids Research, 51(D1): D1373-D1380, 2023.
R. Kneller. The importance of new companies for drug discovery: origins of decade of new drugs. In Nat Rev Drug Discov 9, 867-882 (2010), 2010.
Clayton W Kosonocky, Claus O Wilke, Edward M Marcotte, and Andrew D Ellington. Mining patents with large language models demonstrates congruence functional labels and chemical structures. arXiv preprint S.Ki::2309.08666 2023.
Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alan Aspuru-Guzik. Self-referencing embedded strings (selfies): 100% robust molecular string representation. Machine Learning: Science and Technology, 1(4):045024, 2020.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Facze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. TV." ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arX1v:2411.15124 2024.
Seul Lee, Karsten Kreis, Srimukh Prasad Veccham, Meng Liu, Danny Reidenbach, Saee Paliwal, Weili Nie, and Arash Vahdat. Rethinking molecule synthesizability with chain-of-reaction. arXiv preprint a50iv:2509. 16084, 2025.
Jonathan Lehmann, Daniel J. Blair, and Martin D. Burke. Towards the generalized iterative synthesis of small molecules. Nature Reviews Chemistry, 2(2), Feb 2018. doi: 1038//41570-018 000--0181111.11.1
Biaoshun Li, Mujic Lin, Tiegen Chen, and Ling Wang. Fg-bert: a generalized and self-supervised functional group-based molecular representation learning framework for properties prediction. Briefings in Bioinformat- ics, bbb6)3bb 2023a.
Bo Li. Hao Zhang, Kaichen Zhang, Dong Guo, Yuanhan Zhang, Renrui Zhang, Feng Li, Ziwei Liu, and Chunyuan Li. Llava-next: What else influences visual instruction tuning beyond data?, May 2024a. URL. https://1lava-v thub. o/b httpb//o///2220/0033 25- 1lava next abl inns
14

---

Published as a conference paper at ICLR 2026
Jin Li. Naiteng Wu, Jian Zhang, Honghui Wu, Kunming Pan, Yingxue Wang, Guilong Liu, Xianming Liu, Zhenpeng Yao, and Qiaobao Zhang. Machine somingg-sssssted low-dimensional electrocatalysts design for hydrogen evolution reaction. Nano-Micro Letters, 2023b. doi: 10. 10.1007/s40820-02 0112-5.....
Jin Li, Meisa Zhou, Honghui Wu, Lifei Wang, Jian Zhang, Naiteng Wu, Kunming Pan, Guilong Liu, Yinggan Zhang, Jiajia Han, Xianming Liu, Xiang Chen, Jiayu Wan, and Qiaobao Zhang, Machine learning- assisted property prediction of solid-state electrolyte. Advanced Energy Materials, 2024b. doi: 10. 1002/aemm. 202304480.
Junqi Li. Steven G Ballmer. Eric P Gillis, Seiko Fujüi, Michael Schmidt, Andrea M Palazzolo, Jonathan W Lehmann, Greg F Morehouse, and Martin D Burke. Synthesis of many different types of organic small molecules using one automated process. Science, 347(6227), 2015a. ISSN 0036-8075.
Junqi Li, Steven G Ballmer, Eric P Gillis, Seiko Fujii, Michael J Schmidt, Andrea ME Palazzolo, Jonathan W Lehmann, Greg F Morehouse, and Martin D Burke. Synthesis of many different types of organic small molecules using automated process. Science, 347(6227) 2015b.
Sihang Li, Zhiyuan Liu, Yanchen Luo, Xiang Wang, Xiangnan Hc, Kenji Kawaguchi, Tat-Seng Chua, and Qi Tian. Towards 3d wowecule- interpretation in language models. ArXiv preprint, abs/2401.13923, 2024c. htt.l/tpss//aax...org//////////e
13923
Xiner Li, Limei Wang, Youzhi Luo, Carl Edwards, Shurui Gui, Yuchao Lin, Heng Ji, and Shuiwang Ji. Learning generate 3d molecules via language models with geometry-aware tokenization. In Proc. 2025 International Conference on Muchine Learning (ICML2025), 2025a.
Yibo Li, Yuan Fang, Mengmei Zhang, and Chuan Shi. Advancing molecular graph-text pre- pre-training via fine- grained aligmment. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. pages 1589-1599, 2025h.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Kabeli, Yaniv Shmueli, et al. Evolutionary-scall prediction atomic-level protein structure with a language model. Science, 379(6637):1123 1130, 2023.
Gang Liu, Michael Sun, Wojeicch Matusik, Meng Jiang, and Chen. Multimodal large language models for inverse molccular design with retrosynthetic planning. arXiv preprint arXiv:2410.04223, 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jac Lee. Visual instruction tuning, 2023a.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning. ocr, and world knowledge, January 2024b. BtRLhttts::// htttts://laavavv..ccom
2ea1logg2224--211 11ava 1at---laaaaaaaa/////aaa
Shengehao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Anima Anandkumar. Multi-modal molecule structure-text model text- eased retrieval and editing. ArXiv preprint, abs/2212.10788) 2022. htL//ttss://axxi//
Zhiyuan Liu. Sihang Li. Yanchen Luo, Hao Fei. Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Molca: Molecular graph- saph-languaae modeling with cross-modal projector and uni-modal adapter. arXiv preprini 1...v:2330..179990 2023b.
Zhiyuan Liu, Yaorui Shi, An Zhang, Sihang Li. Enzhi Zhang, Xiang Wang. Kenji Kawaguchi, and Tat-Seng Chua. Reactxt: Understanding molecular" reaction-ship" via craction-comtextuulizee molecule-text arXiv preprint 2rvvv:2405. 14225, 2024c.
prapeccnn pretraining.
Micha Livne, Zulfat Miftahuldinov, Elena Tutubalina, Maksim Kuznetsov, Daniil Polykovskiy. Annika Brundyn, Aastha Jhunjhunwala, Anthony Costa, Alex Aliper, Alán Aspuru Guzik, et al. nachO: multimodal natural and chemical languages foundation model. Chemical Science, 15(22)88380-8 8389,,88922000000 2024.
Steven A Lopez, Edward Pyzer-Knapp, Gregor Simm, Trevor Lutzow, Kewei Li, Laszlo R Seress, Johannes Hachmann, and Alán Aspuru-Guzik. The harvard organic photovoltaic datasct. Scientific data, 3(1):1-7, 2016.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 1rXiv:1711.05101 2017.
Chade Lv. Xin Zhou, Lixiang Zhong, Chunshuang Yan. M. Srinivasan, Z. Seh, Chuntai Liu, Hongge Pan, Shuzhou Li, Yonggang Wen, and Qingyu Yan. Machine learning: An advanced platform for materials development and state prediction in lithium-ion hatteries. Advances in Materials, 2021. doi: 10. 10.1002 1002/amma. 202101474.
15

---

Published as a conference paper at ICLR 2026
Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin Mohr, James M Holton, Jose Luis Olmos, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. Large language models generate functional protein sequences across diverse familics. Nature Biotechnology, 4198):1999-1106 2023.
Rishikesh Magar, Yuyang Wang, Cooper I Lorsung, Chen Liang, Hariharan Ramasubramanian, Peiyuan Li, and Amir Barati Farimani. Auglichem: data augmentation library of chemical structures for machine learning. Machine Learning: Science and Technology, 3(4):045015, nov 2022. doi: 00.1088/2632-2155Wacccc. URL
bittps:/ddt dci. org/ 10......992233 22122223222223///
Arun Maji, Corinne P Soutar, Jiabao Zhang. Agnieszka Lewandowska, Brice Uno, Su Yan. Yogesh Shelke, Ganesh Murhade, Evgeny Nimerovsky, Collin G Borcik, et al. Tuning sterol extraction kinetics yields a renal-sparing polyene antifungal. Nature, 623(7989): 1079-1085, 2023.
Barbara Hikulak-Klucznik Paulina Golghiowska, Alex A. Bayly, and et al. Computational planning of the synthesis of complex natural products. Nature, 588(7836):83-88 2020. doi: (0.1088/41586-000 2855-y.
Xavier Montalban, Karolina FaaseckaSttryzyyii
Jens Kuhle, Pascal Benkert, Douglas L Aroodd Martin S We- ber, Andrea Seitzinger, Hans Guchring, Jamie Shaw, Davorka Tomic, Yann Hyvert, Danielle E Harlow, Martin Dyroff, and Jerry S Wolinsky. Efficacy and safety results after >3.5 years of treatment with the bruton's 's tyro- sine kinase inhibitor evobrutinib in relapsing multiple selerosis: Long-tem follow of phase randomised clinical trial with a cerebrospinal fluid sub-study. Multiple Sclemosis Journal, 5054-5) 2024, doi: 10. I 1177113525585412244883 URL https: 38436271.
PMID:
Thao Nguyen. Kuan-Hao Huang, Ge Liu, Martin D Burke, Ying Diao, and Heng Ji. Farm: Functional group- aware representations for small molecules. arXiv preprint wrXiv:2410.02088 2024a.
Thao Nguyen, Tiara Torres- Floree,, Changhyun Hwang, Carl Edwards, Ying Diao, and Heng Ji. Glad: Syner- gizing molecular graphs and language descriptors for enhanced power conversion efficiency prediction in organic photovoltaic devices. In Proc. 33rd ACM International Conference on Information and Knowledge Management (CIKM 2024), 2024b.
Thao Nguyen. Tiara Torres-Fioc Flores, Changhyun Hwang. Carl Edwards, Ying Diao, and Heng Ji. Glad: Syner- gizing molecular graphs and language descriptors for enhanced power conversion efficiency prediction in organic photovoltaic devices. In Proceedings the 33rd ACM International Conference Information and Knowledge Management, pages 4777-4785, 2024c.
Thao Nguyen, Kuan-Hao Huang, Ge Liu, Martin D. Burke, Ying Diao, and Heng Ji. Farm: Functional group- aware representations for small molecules. In Proc. NAAC12025 Workshop on and Scientific Discovery: Directions and Opportunities, 2025.
Emmanuel Noutahi, Cristian Gabellini, Michael Craig, Jonathan SC Lim, and Prudencio Tossou. Gotta be safe: a new framework for molccular design. Digital Discovery, 3(4):796-8)) -04,4, 2024.
OpenAI, GPT-4o: Large language model. https://opeeai.com 2025a. Accessed: 2025-09-22.
OpenAl. GPT-5 large language model. https::/openai.coom
2025b. Accessed: 2025-09-22.
Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, and Rui Yan. Biot5: Enriching cross-modal integration in biology with chemical knowledge and natural language associations. arXiv preprint arXtv:2310.07276 2023.
Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan. Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning, ArXiv preprint, abs/2402. 2024. htLL/httss:/aarttte org/ abs 2402 17910
Shana Poplack. "sometimes i'll start a sentence in spanish y termino en español" Toward a typology of code-switching. Linguistics, 51(s1):11 1--14441114.1111 2013.
Benjamin Sanchez-Lengeling Jennifer N Wei, Brian K Lee, Richard C Gerkin, Al'an Aspuru-Guzik, and Alexander B Wiltschko. Machine learning for scent: Learning generalizable perceptual representations of small molecules. arXiv preprint a91iv:1910. 10685, 2019.
K. Senior. Gleevec does not cross blood-brain barrier. The Lancet. Oncology, 2003. doi: 10.101661170--205500) 01050-7.
Zhan Si. Deguang Liu, Wan Nie. Jingjing Hu, Wei Wang. Tingting Jiang, Haizhu Yu, and Yao Fu. Data-based prediction of redox potentials via introducing chemical features into the transformer architecture. Journal of Chemical Information and Modeling, 2024. doi: 10. 1e..0//ses.jiim.cc113
16

---

Published as a conference paper at ICLR 2026
Antoine Simoneau, Charlotte B Pratt, Hsin-JJng Wu, Shreya S Rajeswaran, Charlotte Grace Comer, Sirimas Sudsakom, Wenhai Zhang, Shangtao Liu, Samuel Meier, Ashley Choi, al. Characterization of tmg348; a selective, allosteric uspl inhibitor that synergizes with parp inhibitors tumors with homologous recombination deficiency. Molecular Cancer Therapeutics, 2025.
Karan Singhal, Shekoofch Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Hcather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620(7972): 172-180, 2023.
E. Solak and E. Irmak. Advances in organic photovoltaic cells: a comprehensive review of materials, technologies, and performance. RSC Advances, 2023. doi: 10. 1039/d3ra0145444
Henry W Sprucill, Carl Edwards, Khushbu Agarwal, Mariefel V Olarte, Udishnu Sanyal, Conrad Johnston, Hongbin Liu, Heng Ji, and Sutanay Choudhury. Chemreasoner: Heuristic search over large language model's knowledge space using quantum -ehaattt--eemicall feedback. ArXiv preprint, abs/2402.10980, 2024. URL
https://arxiv..rgg abs/ 2402. 10980
Felix Stricth-Kalthoff Han Hao, Vandana Rathore, Joshua Derasp, Théophilc Gaudin, Nicholas H Angello, Martin Seifrid, Ekaterina Trushina, Mason Guy, Junliang Liu, Xun Tang, Masashi Mamada, Wesley Wang, Tuul Tsagaantsooj, Cyrille Lavigne, Robert Pollice, Tony C. Wu, Kazuhiro Hotta, Leticia Bodo, Shangyu Li, Mohammad Haddadnia, Agnieszka Wolos, Rafal Roszak, Cher Tian Ser, Carlota Bozal-Ginesta. Riley J Hickman, Jenya Vestfrid, Andrés Aguilar-Granda, Elena L Klimareva, Ralph C Sigerson, Wenduan Hou, Daniel Gahler, Slawomir Lach, Adrian Warzybok, Oleg Borodin, Simon Rohrbach, Benjamin Sanchez- Lengeling. Chihaya Adachi, Bartosz A Grzybowski, Leroy Cronin, Jason E Hein, Martin D Burke, and Alán Aspuru- Guzik. Delocalized, asynchronous, closed-loop discovery of organic laser emitters. Science, 384 (6697)cead99227, May 2024a.
Felix Stricth-Kalthoff Kalthoff, Han Hao, Vandana Rathore, Joshua Derasp, Théophilc Gaudin, Nicholas H Angello, Martin Seifrid, Ekaterina Trushina, Mason Guy, Junliang Liu, et al. Delocalized, asynchronous, closed-loop discovery of organic laser emitters. Science, 38446697) 2024b.
Felix Strieth- Kalthoff,
Strieth-Kalthfff, Sara Szymkuc, Karol Molga, Alán Aspuru Guzik, Frank Glorius, and Bartosz A Grzy- bowski. Artificial intelligence for retrosynthetic planning needs both data and expert knowledge. Journal of the American Chemical Society, 146(16): 1005-11017, 2024c.
Dagmar Stumpfe, Huabin Hu, and Jurgen Bajorath. Evolving concept of activity cliffs. ACS omega, 4(11):
14360-14368 2019.
Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with tructure-awarc vocabulary. In ICLR, 2024.
Xiaoyu Sun, Nathaniel Krakauer, Alexander Politowicz, Wei-Ting Chen, Qiying Li, Zuoyi Li, Xianjia Shao, Alfred Sunaryo. Mingren Shen, James Wang, and Dane Morgan. Assessing graph-based based deep learning models for predicting flash point. Mol. Inf., (9(6): 1900101, feb 2020. doi: 10. :000//mnf.201900110 URI.
/ttpis://dci crg/ 10.00....... 1002 Fmi f. 201 900101
N. Takayama, N. Sato, S. O' O'Brien, Y. Ikeda, and S. Okamoto. Imatinib mesylate has limited activity against the central nervous system vementement of philadelphia chromosome positive acute lymphoblastic leukaemia due poor penetration into cerebrospinal fluid. British journal of haematology, 2002. doi: 10.1644/j.1366 1365-2141. 2002.03881.x.
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science, arXiv preprint arXiv:2211.09085, 2022.
A. Thirunavukarasu, D. Ting, Kabilan Flangovan, Laura Gutierrez, Ting Fang Tan, and D. Ting. Large language models in medicine. Nature Network Boston, 2023. doi: 10.1038/s41591-000 151-1113---2-248---88
Melanie Trobe and Martin D. Burke. The molecular industrial revolution: Automated synthesis of small molecules. Angewandte Chemie International Edition, 57(16)4412--4214 Mar 2018. doi: 10.10a2/. 201710482.
Thcodore Tyrikos-Ergas Sevasti Agiakloglou, Antomio J Laporte, Wesley Wang, Chich-Kai Chan, Clare E Wells, Christopher K Rakowski, Rachel Hammond, Jia Qiu, Jonathan D Raymond, et al. Automated iterative ne and ec bond formation. ChemRxiv, 2025.
Derek Van Tilborg, Alisa Alenicheva, and Francesca Grisoni. Exposing the limitations of molecular machine learning with activity cliffs. Journal of chemical information and modeling, 62(23):5938-59) 2022.
17

---

Published as a conference paper at ICLR 2026
Edon Vitaku, David T. Smith, and Jon T. Njardarson. Analysis of the structural diversity, substitution patterns. and frequency of nitrogen heterocycles among u.s. fda approved pharmaceuticals. Journal of Medicinal Chemistry, 2014, doi: 10. 10.1021/jm501 100b.
Hongwei Wang. Weijiang Li, Xiaomeng Jin, Kyunghyun Cho, Heng Ji. Jiawei Han, and Martin Burke. Chemical- reaction- aware molecule representation learning. In Proc. The International Conference on Learning Repre- sentations (ICLR2022), 2022a.
Ruheng Wang. Hang Zhang. Trieu Nguyen, Shasha Feng, Hao- Wei Pang, Xiang Yu, Li Xiao, and Peter Zhiping Zhang. Pepthink-rl: Llm for interpretable eyelic peptide optimization with sft and reinforcement learning. arXiv preprint ar 25v:2508, 14765, 2025a.
Sheng Wang. Yuzhi Guo. Yuhong Wang. Hongmao Sun, and Junzhou Huang. Smiles- large scale unsuper- vised pre-training for molecular property prediction. In Proceedings of the 10th ACM international conference on bioinformatics, computational biology and health informatics, pages 429-436, 2019.
Wesley Wang. Nicholas H Angello, Daniel J Blair, Theodore Tyrikos Ergas, William H Krueger, Kameron N S Medine, Antonio LaPorte, Joshua Berger, and Martin D Burke. Rapid automated iterative small molecule synthesis. Not. Syath,, (188))1031-108,, May 2024.
Tyrikos-Ergas,
Xinyou Wang, Zaixiang Zheng, Fei YE, Dongyu Xue, Shujian Huang, and Quanquan Gu. DPLM-2: A multimodal diffusion protein language model. In The Thirteenth International Conference on Learning Representations, 2025b. URL
net Li55
Yifei Wang. Shiyang Chen, Guobin Chen, Ethan Shurberg, Hang Liu, and Pengyu Hong. Motif-based graph representation learning with application chemical molecules. Informatics, volume 10, page 8. MDPI, 2023.
Yuyang Wang, Jianren Wang, Zhonglin Cao, and Amir Barati Farimani. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 4(3):279-287, 2022b.
David Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1 (8)3333-3,, 1988.
David S Wishart, Sagan Girod, Harrison Peters, Eponine Oler, Juan Jovel, Zachary Budinski, Ralph Milford, Vicki W Lui, Zinat Sayeeda. Robert Mah, et al. Chemfont: the chemical functional ontology resource. Nucleic Acids Research, (D(D)))D1220DD1222 2023.
Eric M Woerly, Jahnabi Roy, and Martin D Burke. Synthesis of most polyene natural product motif's using just 12 building blocks and one coupling reaction. Nature Chemistry., 6, 2014, ISSN 1755-4349.
Agnieszka Wotos, Dominik Koszelewski, Rafat Roszak, Sara Szymkuć, Martyna Moskal, Ryszard Ostaszewski, Brenden T Herrera, Josef M Maier, Gordon Brezicki, Jonathon Samuel, et al. Computer-designed repurposing of chemical wastes into drugs. Nature, 604(7907)x68866) 2022.
Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Calch Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenct: a benchmark for molecular machine learning. Chemical science, 9 (2):513-530, 2018.
Xia. Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Li. Mole-bert: Rethinking pre-training graph neural networks for molecules. In The Eleventh International Conference on Learning Representations, 2023.
Hanguang Xiao, Feizhong Zhou, Xingyue Liu, Tianqi Liu, Zhipeng Li, Xin Liu, and Xiaoxuan Huang. A com- prehensive survey of large language models and multimodal large language models medicine. Information Fusion, page 102888, 2024a.
Yijia Xiao, Edward Sun, Yiqiao Jin, Qifan Wang, and Wei Wang. Proteingpt: Multimodal I1m for protein property prediction and structure understanding. arXiv:2408. //66, 2024b.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, al. Qwen2. 5 technical report. arXiv preprint 1511v:2412.15115, 2024.
Geyan Ye, Xihao Cai, Houtim Lai, Xing Wang, Junhong Huang, Longyue Wang, Wei Liu, and Xiangxiang Zeng. Drugassist: A large language model for molecule optimization. Briefings in Bioinformatics, 26(1 )bbbac693,, 2025.
Botao Yu, Frazier N Baker, Ziqi Chen, Xia Ning, and Huan Sun. Llasmol: Advancing large language models for chemistry with large-scale, comprehensive, high-quality instruction tuning dataset. arXiv preprint 299. 2024.
18

---

Published as a conference paper at ICLR 2026
Di Zhang, Wei Liu, Qian Tan, Jingdan Chen, Hang Yan, Yuliang Yan, Jiatong Li, Weiran Huang. Xiangyu Yue, Dongzhan Zhou, Chemllm: chemical large language model. arXiv preprint arXiv:2402 06852, 2024a.
Shichang Zhang. Ziniu Hu, Arjun Subramonian, and Yizhou Sun. Motif-driven contrastive learning of graph representations. arXiv preprint arXiv:2012.12533: 2020.
Xuan Zhang, Limei Wang. Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, Keir Adams, Maurice Weiler, Xiner Li. Tianfan Fu, Yucheng Wang, Haiyang Yu, YuQing Xie, Xiang Fu, Alex Strasser. Shenglong Xu, Yi Liu, Yuanqi Du, Alexandra Saxton, Hongyi Ling, Hannah Lawrence, Hannes Stärk, Shurui Gui, Carl Edwards. Nicholas Gao, Adriana Ladera, Tailin Wu, Elyssa F. Hofgard, Aria Mansouri Tehrani, Rui Wang, Ameya Daigavane, Montgomery Bohde, Jerry Kurtin, Qian Huang. Tuong Phung. Minkai Xu, Chaitanya K. Joshi, Simon V. Mathis, Kamyar Azizzadenesheli, Ada Fang, Alán Aspuru-Guzik, Erik Bekkers, Michael Bronstein, Marinka Zitmik, Anima Anandkumar, Stefano Ermon, Pietro Liò, Rose Yu, Stephan Günnemann, Jure Leskovec, Heng Ji, Jimeng Sun, Regina Barzilay, Tommi Jaakkola, Connor W. Coley, Xiaoning Qian, Xiaofeng Qian, Tess Smidt, and Shuiwang Ji. Artificial intelligence for science in quantum, atomistic, and continuum systems. In Foundations and Trends in Muchine Learning, 2025.
Yu Zhang, Xiusi Chen, Bowen Jin, Sheng Wang, Shuiwang Ji, Wei Wang. and Jiawei Han. A comprehensive survey of scientific large language models and their applications scientific discovery. arXiv preprint arXiv:2406. 10833, 2024b.
Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based graph self- sulf-suupr learning for molecular property prediction. Advances in Neural Information Processing Systems, 34:15870- 15882, 2021.
Ziqiao Zhang, Bangyi Zhao, Ailin Xie, Yatao Bian, and Shuigeng Zhou, Activity cliff prediction: Dataset and benchmark. arXiv preprint arXiv:230..0551,, 2023.
Haiteng Zhao, Shengchao Liu, Ma Chang. Hannan Xu, Jie Fu, Zhihong Deng. Lingpeng Kong, and Qi Liu. Gimlet: A unitied graph-text model for instruction based molecule zero-shot learning. Advances in neural information processing systems, 36:5850-5887, 2023.
Zihan Zhao, Da Ma, Lu Chen, Liangtai Sun, Zihao Li, Hongshen Xu, Zichen Zhu, Su Zhu, Shuai Fan, Guodong Shen, et al. Chemdfm: Dialogue foundation model for chemistry. arXiv preprint arXiv:2401.144 148888 2024.
Yizhen Zheng, Huan Yee Koh. Maddie Yang. Li Li, Lauren T. May, Geoffrey I. Webb, Shirui Pan, and George Church. Large language models in drug discovery and development: From discase mechanisms to clinical trials, 2024. WRRLhhtpss//aaxx v. c abs/ 2409 2409244999444111
Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference on Learning Representations, 2023.
Xiao Zhu, Chenchen Qin, Fang Wang, Fan Yang, Bing He, Yu Zhao, and Jianhua Yao. Cd-gpt: a biological foundation model bridging the gap between molecular sequences through central dogma. bioRxiv, pages 2024-06, 2024.
George Kingsley Zipf. The psycho-biology of language: An introduction to dynamic philology. Routledge, 1936.
19

---

Published as a conference paper at ICLR 2026
A ADDITIONAL RESULTS
A.1 SYNTHESIZABILITT
For the mCLM, all starting materials the building blocks, and the same collection of building blocks used redundantly to make the proposed structures. this analysis, we assume that all the building blocks required for the mCLM approach are available. Given a sequence of blocks A C in our methodology. we can assemble them with the following steps:
Coupling reaction (A and B)
Deprotect reaction (AB)
Coupling reaction (AB and C)
Deprotect reaction (ABC)
Coupling reaction (ABC and D)
Scaling this formula building blocks gives requirement of 2n reactions (synthetic steps).
On the other hand, each molecule generated by MoleculeSTM unique problem for synthesis, and thus cannot leverage the strengths solution.
block-based approach. Each generated target requires a customized synthesis
However, it is important to consider an external metric for verifying the synthesis capabilities of the mCLM. In ta compute validity and synthetic accessibility scores for all baselines, both overall and in terms of specific property modifications. We find that mCLM outperforms all baselines (lower SA better).
Table 3: Synthetic Accessibility (SA) scores with RDKit validity percentages across datasets.
Dataset
SA
FDA
Va3id
S.A
Volid (55)
MolecokSTM Valil(S)
SA
FincMolTex Volid
SA
Valid (55)
GPI5
Valid
SA
Gemini--554
Valid
SA
Volid (5)
Claude
Volid (14)
AMES
2:70
$00LD
2.30
100.0
2.64
16:25
2.52
94.25
3,06
90.95
2.08
72.13
2.92
20:34
2.85
90.15
3:12
93.44
2.50
100.0
2,61
2.67
88.52
2.89
DILI
2.70
100.0
100.0
2.59
2.73
2.70
$8.52
HUS
2.70
100.0
1.42
100.0
2.73
91
2.59
93.44
1.68
87.70
2.62
73.77
2.77
04.26
1.80
90.16
2.56
91.50
PCaP
2.50
100LD
2:45
100.0
2.63
03:44
2.99
94.25
3.81
90.16
2.65
66.00
2.70
62.70
3.87
93:44
2.15
90.
2.70
100.0
100.0
2.59
96.02
79.51
2.77
2.83
91.53
Next, we select the best 3 models and consider computationally expensive but rigorous approach. We leverage Allchemy, which the statc-of-the-at retrosynthesis software, to quantitatively measure this. We randomly sampled a representative set 200 molecules generated by each baseline (and the original FDA molecules). We gave each molecule 30 minutes on a superconputing cluster (which is quite exhaustive) and didn't apply a price limit for substrates (so this is really quite generous for starting materials). Scores, as reported in Table 2 show much stronger overall performance of the mCLM. Here, validity is the percent of syntactically correct molecules as measured by RDKit, synthesizability is percent valid molecules where Allchemy could find a synthetic path, and Makeability is the product of those two metrics. Makeability represents the percent of generated molecules which could be made in the lab.
A.2 EXPANDED TESTING WITHOUT SYNTHESIS GUARANTEES
As a large-scale test, we apply the mCLM to improve all FDA-approved drugs consisting of least 3 blocks (without synthesis guarantees). Note: this is just a confirmatory test, since the mCLM is intended to used on the distribution of synthesis-guarann guaranteed molecules (as shown in the main paper). This amounts to 430 molecular structures and 796 unseen blocks. Since most of these molecules (426/430) contain blocks that were not present in the 1,000 blocks used for training, this presents opportunity find how the mCLM performs out-of- distribution. Results Table show that improvement achieved 5/6 properties. even though the model not seen almost half of the blocks in its vocabulary during training. Specifically, the strictest synthesis-gurranted tokenizer covers 26.7% of compounds in our corpus. For the remainder. rule-based tokenizer is used. Despite this fallback, mCLM demonstrates strong generalization, it successfully incorporated GNN-derived features for the unseen modules.
guaranteed
20

---

Published as a conference paper at ICLR 2026
Table Average pharmacokinetic and toxicity properties of FDA drugs with 3 or more blocks and their proposed modifications. Note, these molecules do not have mouthesi--uuranntes, (l: lower is better, t: higher is better.
Property
AMES Mut. (1)
BBBP (1)
CYP344 Inhib. ())
DILI (4)
HIA (1)
PGP (U)
Mean Improv.
FDA Drug 59.5
37.6
2.0
66.2
93.2
66.0
0.00 %
mCLM
54.0
41.4
1.2
55.5
97.6
68.0
12.87 %
A.3 APPLYING MOLRCULESTM THE "FALLEN ANGELS"
As a baseline comparison, we repeat the fallen angels reasoning experiment using MoleculeSTM (Figure 6 Even though the steps and of MoleculeSTM's modification of TNG348 showed comparable property changes mCLM. MoleculeSTM encounters molecule validity problems: it generates syntactically incorrect molecule on step of Evobrutinib and on step 3 of TNG348.
Decrease DILI
Decrease PGP
Decrease AMES
Drug:
Evobrutinib
Target:
BTK
INVALID
INVALID
INVALID
Reason
Failure:
DILI:
82.7
Liver
Toxicity
AMES: 77.2
Drug:
TNG348
Target:
INVALID
USP1
Reason for
Failure:
DILI:
84.8
DILI: 71.5
DIL:: 67.3
Liver
BBBP:
868P: 100.
ВВВР:
Toxicity
CYP3a4: 0.35
CYP3a4: 4.28
CYP3a4: 3.22
Decrease DILI
Increase BBBP
Decrease CYP3a4
Figure 6: Examples of fallen angel property modification using MoleculeSTM.
B CRITICAL CHEMICAL REASONING ALGORITHM
Algorithm Critical Chemical Reasoning for Molecule Design mCLM
1: Input: Initial molecule My
2: Output: Refined molecule M
3: Initialize M + Mo
4: while True do
5: Enumerate over functions to improve in M (e.g., metabolism, drug-induced organ injury,
blood-brain barrier penetration)
6: if No clear objective remains for improvement then return M
7:
8: elseSelect property which requires most improvement.
9:
end
10: mCLM generates a candidate modification M' by replacing, adding, or removing building
blocks M
11: Evaluate M' with respect to desired functions
12: end while
21

---

Published as a conference paper at ICLR 2026
Training Corpus
Molecule property/
Imatinbb
& tyrosine kinase inhibitor that speciticaly torgets the 8CR-ASL protein...
function description
Molecule proposal
To enhonce selectivity and patency against resistont concer celi lines, we can use this maleculer
Molecule
Generate molecule that has higher blood brain barrier permesblitt than (MOI)
mag MMOKi
optimization
Answer: (MOL) - 5
D2 [/MOL]
aa papaac ac?
Synthesis prediction
last step synthesis reaction of imatonlb is:
Synthesis Tokenizer
Text Tokenization
Modify Porcetrupib sthy quinal -I- carbooylate mininise presouse
Molecule Pesenizatinn
Text Tokens
Molecular Graph Tokens
Fully Synthesizable Vocab
High effeary
Large-scale Pretraining Vocab
law towicity
Retrieval-basel molecule generation flexible
Synthesis- Guaranteed Synthesi-Guaranteedd Generation
Function-aware Vocabulary Selection
inference time vocabulary
Figure Overview of Data Creation.
C PRETRAINING DATA MIXTURE
To train the model, our goal is to cover wide variety of different molecular functions and classes of molecules. To do consider wide variety of data sources. We three existing instruction datasets: SMolInstruct Yu al. (2024), Tulu-3 (Lambert al. 2004). and the Biomolecular text portion of Mol-Instructions (Fang) ct al. 2023). SMolInstruct is used to cover the standard set of chemistry LLM tasks. We supplement this with oon-overlapping portion of Mol- hootrusstuctsons. Finally, we include Tulu-3 to preserve general reasoning capability.
C.I PROPERTY-BASED SYNTHETIC INSTRUCTION DATA GENERATION FOR CONTRASTIVE LEARNING
Additionally, we augment this data mixture with significant amount of synthetic data created using real- world property datasets. Given our datasets, create additional question datapoints for both regression and classification tasks, such "Is this molecule <SMILESS> </SMIL.ES> blood brain barrier permeable? Yes". also consider the opposite direction: property to molecule and multiple propertics to molccule, and unconditional molecule generation. The templates used for constructing this data were created using GPT-4o; 50 were originally generated and then bad templates were removed by hand. Further, we augmented each data point with description of the property, also written by GPT-4o.
addition these tasks, we also consider the molecule optimization task described in DrugAssist Ye 2025) (c.g., given molecule A, improve property X). However, their approach has fundamental limitations due to the reliance on oracle models. These models may be out-of-distribatiion for molecules within our dataset, and low-quality models may propagate errors into training data. To address this issue, we consider "activity cliffs" (Stumpfe al. 2019 Zhang
2023 within existing property datasets train the model for molecule optimization. Activity clifts are two molecules which are structurally similar have large differences in given property. We opt to base our definition of activity cliff on Murcko scaffolds (Bemis and Murcko [996). This methodology, which is widely used in medicinal chemistry, extracts the core subgraph, or scaffold, of molecule. We found that this approach was much more computationally feasible on large-scale molecule datasets than approaches like fingerprint similarity (Bajusz
2015
22

---

Published as a conference paper at ICLR 2026
Given nominal data (e.g.. toxic, kinase inhibitor, banana smell), look for a pair of molecules that have the same scaffold only one molecule has the desired property. Given numerical data (c.g., solubility, power conversion efficiency, HOMO-LUMO gap), we instead look for molecules which have a property difference I standard deviation. This forms the positive-negative same-scaffole portion of our data. We also consider pairs of molecules that have different scaffolds but the same property. Here, our goal is to teach the model to consider function over structure. As example, may ask "Propose molecule with different structure than [MOL]] [/MOL] that demonstrates anticoagulant properties. [MOL] ... [/MOL]". types, we use oracle models instead of ground truth data. Please see Appendix D.2.3 for more information.
select number of property
C.2 DATA FILTERING
Due to the large size of our dataset, we filter our data to only keep the most frequent 100k (SOk end blocks and 50k mid blocks). 10k (5k and 5k), and 1000 blocks (500 and 500) for experiments. This is because there are far fewer molecule tokens compared to text (each training sample only has up to about 10). Due to Zipf"s s law (Zipf) 1936), the full dataset, single molecule token may only appear in one example the entire training corpus. Without considerably scaling the compute budget, understanding this additional block is difficult. Overall, this filtering enables us to learn more efficiently and requires less memory resources for training the model, and allows us to better test the model architecture. Please the section on training details for more information in Appendix [aadddiExEandd and instruction type for each subset of the data. Table dable/oggives gives breakdown the molecule data each subset of the data. Note that not all molecules in the "Full Data" can be represented with blocks, so we have a subset for "Only Blocks" molccules, and then molecules consisting of the most frequent "Top 100k" blocks, "Top 10k" blocks, and "Top 1000" blocks. Scaffolds are the number of unique Mureko scaffolds (Bemis and Murcko [996). "Caps" are end blocks, while "Mid" are middle blocks. "Synthesis-awarre" is the number of synthesis guaranteed blocks out "Total". Data splits are subdivided into their original source: our constructed instruction data or molecule instructions from SMolInstruct.
Appendix for source property datasets. Table gives breakdown data quantity by
synthesis-guaranteed
Table 5: Dataset categories and their respective sample counts.
Data Type
Full Data
Only Blocks
Top 100k
Top 10k
Top 1000
All
36,641,170
20,910,431
14,823,280
6,195,903
1,054,12 124
Existing Data Sourees
3,117,841
1,208,365
1,049.473
537,674
149,994
SMolInstruet
2,124,738
215,262
56,370
14,242
2,300
Tulu-3
939,343
939,343
939.343
469,672
93,934
Instrucciiinn Biomol. Text
53,760
53,760
53,760
53,760
53,760
Synthetic Real Data (Ours)
21,290,353
9,294.434
7,711,548
3,960,633
705,210
Classification
5,612,215
3,771,038
3,192.583
1,337,619
223,341
Molecule Generation
535.317
316,630
245,034
150,001
27.393
Positive Negative Same Scaffold
3,795,820
113,390
65,566
16,286
2,007
Positive Positive Different Scaffold
660,925
477,976
256,435
191,720
33,579
Property to Molecule
6,676,957
3,125,097
2,878,012
1,639,322
307,780
Multi-Pro Multi-Propprtty to Molecule
572,505
556.107
533,647
423,089
78,793
Scaffold+Propert Molecule
705,833
405,497
226,696
80,448
13,078
Regression
2,730,781
528,699
313,575
122,148
19,239
Synthetic Oracle Data (Ours)
12,232.976
10,407,632
6,062,259
1,697,596
198,920
Classification
2,077.327
1,778.874
1,058,754
299,444
35,168
Positive Negative Same Scaffold
72,763
58,645
25,035
4,935
469
Scaffold+Proporty Molecule
1,169,407
925,055
453,087
113,049
13,642
Property to Molecule
13,257
13,254
8,841
906
2
Regression
8,900,222
7,631,804
4,516,542
1,279,262
149,639
D SYNTHETIC DATA GENERATION
D. TASK FORMULATION FOR MOLECULAR DESIGN
To train modular chemical language model (mCLM), we generate synthetic data that reflect realistic scenarios encountered in molecular design. These include tasks relevant to drug discovery and organic photovoltaic (OPV) materials design. Our goal is to equip the model with the ability to understand and reason over molecular representations and natural language prompts across diverse set of tasks.
Drug Discovery Tasks: In the context of drug discovery, chemists often engage in iterative and multi-objective optimization processes, where molecules are evaluated, modified, generated hased various physicochemical
23

---

Published as a conference paper at ICLR 2026
Table 6: Breakdown of molecule data by dataset category.
Molecules
Molecule
Subset
Source
Total
Tokenized
Untekenized
ScatTolds
Cap
Mid
Total
Synthesis-Awar
SMolinstruet
1,951,205
1,566,030
385,175
510,464
Full Dara
Our data
6,160,565
5,270,962
864,212
1,109,622
Tooal
7,994,305
6,787,879
1,178,957 0
1.537,422
SMolInstruet
459,910
459,910
145,6
157,204
59,681
216,885
50,788
Only Blocks
Ours
3,830,543
3,830,543
705,641
175,903
189,770
Total 4,220,604
4,220,604
0
799,267
598,470
203,810
802,280
214,431
SMollnstruet
146.079
146.079
0
49,743
22,871
14,809
37.680
10.930
Top 100k
Ours 2.345.519
2,345.519
0
429,
49,941
49,780
99,721
27.823
100,000
MMolInstruet
37,686
37,686
0 12,373
4,130
2.534
6,664
2,618
Top
821,238
821,238
4,998
9,998
3,468
Total
848,674
848.674
0
153,144
5,000
5,000
10,000
3,487
SMollnstruet
4.867
4,867
0
1391
477
359
836
420
Top 1000
Ours
109.554
109,554
0
19,639
500
500
1,000
432
12,6657
20,101
and pharmacological properties. These tasks typically involve querying for specific properties. modifying structures to meet certain design criteria, generating novel candidates that satisfy given constraints. We consider the following tasks that chemist may perform:
Texl-to-Molecule Generation: Generate molecule from textual description of its structure or properties.
Property Prediction: Given molecule and property of interest, predict whether the molecule
possesses the property (binary classification) the quantitative value of the property (regression).
Molecular Optimization: Modify given molecule to satisfy or improve specific property.
Generation: Given scaffold and target property, generate molecule that satisfies both the structural constraint and the property constraint.
To support broader applications of mCLM beyond drug discovery, we also incorporate tasks relevant to organic photovoltaic (OPV) material design. OPVs are emerging class of lightweight, flexible materials used for solar energy harvesting, where the power conversion efficiency (PCE) the primary performance metric. A typical OPV device composed molccule. The donor responsible for absorbing sunlight and generating excitons (electron-hole pairs), while the acceptor facilitates charge separation and electron transport. The chemical compatibility and electronic alignment between the donor and acceptor molecules critically influence the resulting PCE. To enable learning this domain, define the following OPV-specific tasks
OPV Material Design Tasks:
donor moleculc and an acceptor
PCE Prediction: Predict the PCE of given donor-acceptor pair.
Donor/Acceptor Completion: Given a donor (or acceptor) and target PCE, generate the comple- mentary component (acceptor donor) that achieves the desired performance.
Constrained Completion with Scaffold: Generate donor acceptor molecules that match given scaffold and achieve target PCE.
D.2 INSTRUCTION TUNING DATA GENERATION
D.2.1 PROMPT-ANSWER TEMPLATES
To generate instructional data, construct pool of question and answer templates for each task. These templates include multiple paraphrased variants introduce linguistic diversity and improve the model's generalization capability. During data generation, a question template is randomly sampled from question set and populated with spepiccc content. Likewise, a corresponding answer template is sampled from the answer set to form complete prompt response pair. For example:
Question Template:
"Given la molecule] and (a property of interest], modify molecule to achieve [desired property value]."
Answer Template: "The [property] of [molecule] [value]."
This templating strategy allows to produce large number of diverse, semantically equivalent training instances that support instruction tuning across multiple molecular design tasks.
24

---

Published as a conference paper at ICLR 2026
D.2.2 LABEL SOURCES FOR INSTRUCTION TUNING DATA
To enable diverse and meaningful pretraining for mCLM, incorporate both experimentally derived and model-generated labels, covering a broad spectrum of molecular properties critical to chemistry, pharmacology. and materials science. This qual-labeling strategy allows the model to learn from abundant low-level molecular descriptors while also reasoning over high-level functional and biological endpoints.
Low-Level Molecular Properties from ChEMBL2 For foundational chemical deseriptors, we leverage the ChEMBL .25 datahase-a, comprchensive hioactivity resource containing approximately 2 million compounds with rich structural and physicochemical annotations. ChEMBL25 serves abundant and reliable source of labels for low-level properties that are widely used cheminnormmtiis pipelines. From this corpus, we select a core set of descriptors that are most informative for molecular design: Hydrogen bond acceptors (HIBA), Hydrogen bond donors (HBD), LogP sectaol-watter partition coefficient), Molecular weight (MolWt), Number of aromatic rings, Number of rotatable bonds, Topological polar surface area (TPSA). These deseriptors inexpensive compute and provide critical insights into molecular solubility, permeability, and synthetic feasibility them essential for early-stage screening and property-basee filtering.
High-Level Molecular Properties via Oracle Labeling: In addition to low-level properties, we aim to expose the model to high-level functional endpoints that capture complex biological phenomena. Such endpoints are central pharmacokineties, drug safety, and efficacy, but they rarely available in large quantities due to the high cost of experimental validation. Consequently, labeled datasets for these tasks are limited in size and diversity. To address this challenge, we employ the oracle ensemble models to generate synthetic labels for a curated set of ADMET tasks.
D.2.3 ENSEMBLE ORACLE MODEL:
To generate high-quality synthetic labels for downstream tasks, we construct oracle models focused ADMET property prediction. The oracle models we use for ADMET assessment are state-of-the fi-thh-rrt, perfoomrrg models trained on experimental henchmark datasets with strong reported correlation to assay measurements, as shown in Table 7 below. This sctup is highly meaningful in computational drug discovery (especially compared to rougher and less meaningful QED scores) and is designed to emulate digital screening pipelines used in industry.
We select tasks from the Therapeutics Data Commons (TDC) benchmark Jusing following criteria:
Relevance Drug Discovery: The task must reflect critical aspect of drug efficacy toxicity.
Predictability: The task must be reliably predictable using existing models. Specifically, we evaluate all 22 ADMET-related classification tasks in TDC and retain only those where standard models achieve under the ROC curve (AUC) greater than 0.80. This ensures the synthetic labels sufficiently accurate for training purposes.
Based these criteria, we select six tasks:
AMES (mutagenicity).
BBBP (blood-brain barrier permeability).
CYP3A4 inhibition (metabolism).
DILI (drug-induced liver injury),
HIA (human intestinal absorption).
PGP (P-glycoprotein substrate classification).
TDC provides predefined scaffold-based data splits with 8:1:1 ratio for train, validation, and test sets. This splitting strategy ensures that structurally dissimilar compounds are separated across subsets, encouraging generalization to novel scaffolds.
Although TDC provides leaderboards for these tasks, many top-performing entries lack reproducible code working implementations. For instance, the authors of one of the top submissions explicitly acknowledge on Gut that their code is not runnable Therefore, opt to use robust foundation models FARM. ChemBERTA-2, and a GNN ensemble learning.
FARM Nguyen et al. 2024a): A SMILES-based BERT model trained with functional group-aware tokenization.
25

---

Published as a conference paper at ICLR 2026
ChemBERTA-2
Ahmad et al. 2022 A large-scale transformer model trained millions of
canonical SMILES sequenees.
GGNN [Bdwards al. 2024a): A graph neural network trained molecular graphs with atom- and
bond-level features.
To build the ensemble, we use each model as feature extractor. The extracted features concatenated and passed through fully connected layer for final prediction. This ensemble approach is stacking, where multiple base learners feed into a meta-leamer. For each task, we select a threshold that maximizes the Fl score on the validation set. This threshold is then used to binarize the predicted logits into class labels. The performance of our ensemble model across the selected tasks is summarized in Table
Table 7: Performance (AUC) of individual models and the ensemble across six selected ADMET tasks.
Model
AMES
Pgp
DILI
BBBP
CYP3A4
HIA
FARM (Nguyen et al. 2024a)
0.88
0.89
0.79
0.94
0.88
0.92
GNN (Edwards et al.
2024a)
0.75
0.78
0.86
0.79
0.80
0.81
ChemBEKIa- (Ahmad et a. 2022)
0.86
0.89
0.81
0.93
0.86
0.99
Ensemble
0.89
0.91
0.84
0.93
0.89
0.99
E TRAINING PROCEDURE
We employed Qwen2.5-3B Yang et al. followed the training procedure from LLa Va (Liu the starting LLM for building the mCLM. Generally, we
2024
al. 2024a). We used a two-layer MLP with PReLU activation (He et al. 2015) as an adapter into the LLM input/output from the GNN. We selected an initial learning rate of Ic-5 for the full model and 1c-6 for the adaptor and LM heads. Further, we used a cosine annealing schedule with a minimum of le-6 and 2000 linear warmup steps; AdamW (Loshchilov and Hutter 2017) optimizer was employed. The model was trained on A100 80GB GPUs in bíloat preciston.
2202a 2024b
We found that the model learned the molecule tokens much slower than the text (there usually 10x difference in loss value). Molecule tokens are rarer and show up less in the training data. Because of this, we decided to separate the molecule classifier head and the language classifier head, We used a standard autoregressive language modeling loss for both, and we averaged these two losses for the final loss value. The main part of training experiments focused on minimizing the molecule loss, since the text loss Further, we found PEFT (Hu et al. 202T) was not sufficient adapt to molecules, so full finetuning was required. Roughly 10-50 examples from each synthetic (data source, task) pair were put into a validation set.
easy optimize.
To initialize the GNN weights, we employed the MoICLR (Wang et al. 2022b) unsupervised contrastive learning technique. We used AugliChem (Magar et al. 2022) for the augmentations: random atom masking, random bond deletion, and motif removal. One of these augmentation was selected uniformly at random for each data point. The GNN was initialized using a hatch size of 128 and Ir le-4 with a cosine schedule. The model was trained on all 800k blocks in the full data until convergence on validation set. We tested embedding dimensions between 16 and 4,096 and found 128 dimensions to be sufficient while minimizing total memory cost. This necessary because we stored the entire embedding matrix in GPU memory, which was much faster, but consumed about 20GB VRAM. Doing allowed us to train without the GNN during our pretraining process, which is considerably more efficient. We note the GNN can then be finetuned along with the rest of the mCLM during finctuning to new types of molecules or specific tasks, While we did consider a sampled softmax to train the mCLM, we found this to limit the learning of the model.
For training the mCLM, used two stages for pretraining. First, we trained for epoch with everything frozen except the adaptor, to allow the adaptor to adjust to the LLM representation space. For the second stage, we trained for 5 epochs with only the GNN embeddings frozen. As discussed in the training data mixture section we used the most frequent 1000 building blocks as our vocabulary.
After pretraining, we finetune the mCLM to standardize it's outputs for our experiments. During pretraining. we train for robustness by using wide variety of responses (e.g.. g.. for BBBP prediction we might respond "It restricted from entering the central nervous system" instead of 'No'). For finetuning. we train with standardized responses for our desired tasks (c.g.. "Generate a molecule that has higher blood brain barrier permeability than [MOL]... ... \/MOL]", "[MOL].. ... [MOL]". Due to our downstream tasks, we finetuned exclusively on the molecule optimization task for 5 epochs over 100k examples for each property. We trained using the same procedure as the pretraining stage, hut we selected the hest model using validation loss.
26

---

Published as a conference paper at ICLR 2026
F TOKENIZER DETAILS
F.I SYNTHESIS- SYNTHESIS-GUARRANTEDD TOKENIZER DETAILS
The synthesis-guand guaranteed tokenizer disconnects the molecule only at bonds that can be formed by a pre- determined small set of reactions, preferably only those that can be performed in automated manner. For instance, if amide bond formation defined available, the tokenizer will be able disconnect amide bonds in the molecule of interest. Up to this point, the protocol is synonymous with classical computational retrosynthesis. but there is a fundamental differencc. Namely, the sets of reactions suitable for automated synthesis is very limited in fact, state-of of of-thhe-aeeee art synthesis machines utilize only three types of disconnections (amide bond formation, well as Suzuki and Buchwald Hrcchaad-Hartwig couplings). This places very stringent requirements the groups that be present in the disconnected blocks for instance, when the disconnection (say, Buchwald-Hartwa coupling) yields an amine functionality on one of blocks, this block cannot contain any groups that during the anticipated uses of this block would present synthetic incompatibility. In the most trivial case, the block cannot contain another unprotected amine because after the Buchwald- Harttwi disconnection, the block would feature amines which, in turn, would present competing reactive sites (in the Buchwald Harttwg synthesis also the formation amide bonds). Therefore, the tokenizer performs retrosynthetic operations while simultancously checking if they do not lead to blocks with functional groups presenting competing reactivities. Only disconnections avoiding such problems are allowed. This then guarantees that when the corresponding blocks are used to make other molecules, they give only the selective synthesis outcomes. The tokenization process maintains the integrity of the molecules: each molecule block sequence, whether produced by tokenization or generated by mCLM. Each tokenized building block contains placeholder atoms (e.g., [1*], [2*]. (3*]) that encode connection points. These placeholders allow us to preserve the full structural integrity of the original molecule, enabling lossless reconstruction from sequence of tokens. For example, shown in Figure [ the drug imatinib with SMILES string
reconstructed exactly from the building-
Ca1 cac (ccl I Nc2ncec (n2) #3cccnc3) NC (-0) c4cce (cc4) CNSCCN (CC5)
is tokenized into:
13 eelccene""
|2+ cleene [N]1+1]n1*
[1+ Nelcce a (1) (12211)"1,
[ clece (CN2CCN (C) CC2) cc1"
Here, [3*] connects to [2* [1*] connects to [2*), and so on. This process is fully deterministic and reversible. The placeholder atoms and their positions are also encoded by the GNN encoder, allowing the model to reason about both structural and functional context during generation.
Following the connection rules embedded in the placeholders, the original molecule be fully reconstructed with the following steps:
1. ["[3)]ccccccc1", [12+]clcemt [N[1+])n1]" "[1+]Neinced (-c2ccenc2) n1
(following the rule [3+] connects to [2+].
2. ["[1+]NeInccc (ecelnnccccc2ccannee (-c2ccenc2) n1", "[1+]Nclcee (C) ([2+])
+ "[cc]N (C) (ccccnn))))))))))))ccoo c1
(following the rule [1+] connects to [2+]]-
3. ""1i*/Neeeed (Mecsss
" "[3x]C cleee (CN2CCN (C) CC2) cel)
"Celece (NC (=0) c2ece (CN3CCN (C) CC) co2) celNeeincc (-c2cccne2) n1"
(following the rule [1+] connects to [3+]).
F.2 MOLECULE TOKEN EXAMPLES
As discussed in we pretrained our GNN encoder using MoICLR (Wang et al. 2022b on all $00k blocks in our full pretraining dataset. Figures 8 13 shows six examples of random blocks and their 5 nearest neighbors in dataset. The similarity of the blocks and their neighbors shows the high quality representations which were obtained.
27

---

Published as a conference paper at ICLR 2026
Gound Tech
Score 0.75
Scancc 333
Scow, 8.726
Seow, 844
Figure 8: Example token from pretraining dataset. Similarity above each molecule is ECFP4 Fingerprint Tanimoto similarity from the ground truth molecule.
Geand Teds
Scor: 593
0.21
Figure 9: Example token from pretraining dataset. Similarity above each molecule is ECFP4 Fingerprint Tanimoto similarity from the ground truth molecule.
28

---

Published as a conference paper at ICLR 2026
Gound Tech
Score 0.306
Scancc 553
Scaw, 2462
Hh
Figure 10: Example token from pretraining dataset. Similarity above each molecule is ECFP4 Fingerprint Tanimoto similarity from the ground truth molecule.
Geand Teds
Grar
Scan: 843
0.310
Scave
Figure 11: Example token from pretraining dataset. Similarity above each molecule is ECFP4 Fingerprint Tanimoto similarity from the ground truth molecule.
29

---

Published as a conference paper at ICLR 2026
Ooend Tech
Score 0.30
Scancc 58?
Scw.
0.25
Scow,
Figure 12: Example token from pretraining dataset. Similarity above each molecule is ECFP4 Fingerprint Tanimoto similarity from the ground truth molecule.
Ooandd
Scor: 8055
Figure 13: Example token from pretraining dataset. Similarity above each molecule is ECFP4 Fingerprint Tanimoto similarity from the ground truth molecule.
30

---

Published as a conference paper at ICLR 2026
G
SOURCE DATASETS AND DATABASES
Leffingwell odors
Sanchez-l_nnellnng
2019
BACE (Wu ct al.
2018)
Flashpoint (Sun
2020
MUV Wu ct al.
2018
Tox21 Wu
2018)
AMES (uang et al.
2021
Bioavailability (Huang
2021
Caco2 (Haag et al.
2021
Carcinogens [uang et
cav3 t (Hanng et al.
2021
(Huang et al.
elearance hepatocyte
(Huang al.
202
CYP1A2
Huang et al.
202
CYP2C9
(Huang al.
2021
CYP2D6
(Huang et al
2021
CYP3A4
(Huang et al.
2021
DILI (drug induced liver toxicity) (Huang et al.
2021
Half life (Huang al
2021
bERG (Huang et al. 2021
HIA (human intestinal absorption) Huang al.
2021
Hydration free energy (Huang et al.
2021
kenq2 (Manng al.)
2021
LD50 (Huang et al.
202
Lipophilicity (Wu
2018
m muscarinicc (Huang et al.
2021
OPV data (Nguyen et al. 2024b
Lopez et
2016)
orexin_receptor Huang al.
2021
PAMPA NCATS (Huang
2021
Broccatelli (Broccatelli et al.
2022
potassium ion channneanlllll (Huang
2021
PPBR (Plasma Protein Binding Rate)
Huang al. 2021
Pubchem logP
(Kim et al.
2023
SARSCoV2_3CT _CLL Huang et al.
2021
SARSCoV2_ Huang
2021
serine_three fainn_thrcomine_kiasse Huang al.
202
Skin Reaction (Huang et al.
2021)
Solubility_AqSooll (Huang al. 2021
tyrosyl-dna phw.hpoaapphsphooeesttreeeee (Huang et al. 2021
VDss (volume of distribution steady state) Huang et al. 2021)
Molecule Property Cliff Datasets (30+ datasets)
(Van Tilborg et al. 2022)
Chemical Function (CheF) Kosonocky al.
2023
ChemFOnt: the chemical functional ontology resource WWshhart et al. 2023
Pubchem properties (Kim al. 2023
FreeSoly (Wu et al. 2018
QM8 (Wu al. 2018
31

---

Published as a conference paper at ICLR 2026
QM9 (Wu al.
2018)
Thermosol (Wu ct al.
2018
ESOL (Estimated SOLubility)
WW et al.
2018)
Lipo et al.
2018
BBBP (Gaulton al.
2012
ClinTox (Wu et al. 2018
HIV (Wu al. 2018
SIDER (Wu ct al.
2018
Forward synthesis (USPTO) Yu al.
2024)
Retrosynthesis (Yu al.
2024
CheBI-20 (Edwards et
2021
L+M 24 (Edwards et al.
2024c
HBA Gaulton al.
2012)
HBD (Gaulton et al. 2012
MolWt Gaulton al.
2012)
NumAromaticRings (Gaulton et al. 2012)
rotatable bonds (Caulton al.
2012)
TPSA (topological polar surface area) GGulton et al. 2012)
H ADDITIONAL BASELINES
We conducted additional comparisons with two relevant baselines: DGAE (Boget et al., 2024) (a discrete graph autoencoder using vector quantization) and HierVAF (Jin al. 2020) (a hierarchical VAF for molecular graph generation). Property
unique HierVAF. model trained individually for each property using data from the mCLM corpus, giving an unfair advantage per-property optimization, while mCLM operates via instruction prompting. Interestingly. HierVAE performs quite well on synthesizability; we believe that to some extent. this is because Hier is trained using molecules constructed from mCLM tokenizer outputs. Thus, it is able take advantage of the distribution of molecules created by mCLM preprocessing, which likely increases it's synthesizability performance. Also interestingly, we find a disagreement between the less less-rigorous SAscore and actual retrosynthesis metrics, where mCLM outperforms significantly on the more grounded retrosynthesis score. We speculate this is because SAScore prefers simpler molecules those with simpler substructures, whereas retrosynthesis tools evaluate actual synthetic feasibility rather than only based structural patterns. However, synthesizabilityy is still small. Notably, Allchemy may also simply too strong the molecules in this distribution, which reduces the dynamic range of the metric. These specific FDA FDA-approved drugs may be too simple, so the dynamic range of molecules predicted synthesizable by Allchemy is quite restricted, For example, even models like MoleculeSTTM have 90% synthesizabillty when editing these molecules. Future work may need compare models on more difficult distributions of molecules increase the dynamic range.
are in Table and sythesizability scores in Table
that the difference in
Additionally, we find that VierVAE is effective for optimizing properties (Table B), especially properties that already have good scores, such as CYP3A4 and HIA. Still, mCLM outperforms it on average.
Table 8: Comparison of pharmacokinetic and toxicity property scores between mCLM and HierVAE. Note that HierVAE is fine-tuned separately for each property, whereas mCLM is used in instruction- following setting.
Model
AMES (L)
BBBP
CYP3A4 (1)
DILI (L)
HIA (1)
PGP (L)
Avg, Improv.
HierVAE
48.2
66.4
1.2
53.1
99.7
64.3
10.5%
mCLM (Ours)
44.4
85.2
1.4
53.7
98.99
64.4
15.0%
We would also like explain the technical differences between mCLM and vector-quantized methods, and why we adopt our current approach instead a quatti-quantizatiinn technique. Vector-quantized autoeneoders (e.g.. VQ- VAE) are commonly used to build discrete vocabularies for downstream models. However, in the context of molecular design, these learned codebooks lack chemically grounded guarantees. In contrast, our synthesis-errffeed vocabulary consists of synthesis- verified building blocks, derived from retrosynthesis constraints and domain knowledge, ensuring that all generated molecules are compatible with automation-frient frienddl synthesis. Further,
32

---

Published as a conference paper at ICLR 2026
Table 9: Synthetic accessibility (SA) (Etl al. 2009), validity, and retrosynthetic results for
HierVAE. Synthesizability is the percent of valid molecules where retrosynthetic route was found.
Makeability is the overall percent of generations which can be synthesized (Makeability = x
Synth.).
Model
SA (4)
Validity (%)
Syathesizability (%)
Makeability (%)
FDA
2.70
100.0
98.11
98.11 18
mCLM (Ours)
2.43
100.0
98.23
98.23
Hie
2.35
99.70
95.83
95.55
applying vector quantization in this domain risks collapsing chemically distinct structures into the same token, reducing interpretability and limiting extension to blocks inference- time. Instead, encoder pretrained on chemical properties produce semantically meaningful embeddings molecular blocks, which are aligned with a language model via adapter layers. This design allows mCLM to retain both chemical fidelity and generation flexibility, while supporting plug-and-play extensions and instruction-based editing. In Table TO we compare synthesizability of the DGAF and mCT.M M the QM9 dataset. Here, leverage an existing DGAE model trained on QM9, released use synthesis-guaranter QM9 blocks. In this experiment, the difference is quite striking- the use our specialized blocks instead a codebook nearly doubles the syathesizability of generated molecules.
GGN--baee
the authors (Boget al. 2024). The mCLM restricted
Table 10: Synthesizability comparison using Allchemy on QM9 molecules.
Model
Validity (%)
Synthesizability (%)
Makeability (%)
SA Score (L)
DGAE (VQ-based)
100,0
62.0
62.0
4.46
mCLM (Ours)
100.0
100.0
100.0
2.23
ABLATION STUDIES
As detailed in the main text. we performed two ablations of the mCLM block encoding process. First, we trained a SMILES based version of the mCLM without GNN (No GNN), still using Qwen-3B backbone. Sccond, conducted test of the mCLM using blocks extracted from the BRICS algorithm instead of from our tokenization method (No Synth. Tokenizer). These showed significantly degraded property modification mCLM.
capabilities compared
We also did an ablation study of using Llama-3.2-3B as the backbone for mCLM. We find that, as also noted by others in the community, with the same backbone size, Qwen works better when fine- tnne-tuned on downstream tasks than Llama. Specifically, find evidence that the Llama-3.22 3.2-3BB mCLM is overfitting compared with Qwen-2.5 2.5-2 3BmMLMM evidenced by lower training loss but higher validation loss (3.702 v.s. 3.620). We also evaluated the property optimization capability L.ama-3.2-3B-mmC
as follows:
Table 11: Comparison of pharmacokinetic and toxicity property scores between using Qwen-2.5-3B and Llama-3.2-3B as backbones.
Model
AMES
BBBP
CYP344
DILI
HIA
PGP (L)
Avg. Improv.
mCLM (Llama-3.2-3B)
48.2
59.4
59.0
98.1
63,9
2.82%
mC1 M.M (Qwen-2.5-3B)
44.4
85.2
1.4
53.7
98.99
64.4
15.0%
mCLM (No GNN)
5044
46.0
2,5
50.2
97.9
71.7
-7,53%
mCLM (No Synth. Tokenizer)
48.7
55.0
2.9
54.9
98.3
68.3
-19.0%
33


Reviews:
[{"review_id": "Ku6VaSFORh", "reviewer": "Reviewer_k9Sm", "summary": "This paper presents mCLM, a modular chemical language model designed for universal molecular understanding across textual, graphical, and structural modalities. The framework integrates multiple specialized modules—each trained on different molecular representations (e.g., SMILES, graphs, spectra)—and fuses them through a shared latent alignment layer. The authors emphasize scalability, interpretability, and extensibility, demonstrating competitive results on property prediction, reaction reasoning, and cross-modal translation benchmarks such as MolLangBench and MoleculeNet.", "questions": "How does mCLM handle conflicts between representations (e.g., inconsistent SMILES vs. graph encodings)?\n\nCould the modular framework support plug-and-play extensions for new data types (e.g., protein–ligand complexes)?\n\nHow stable is the latent alignment layer during joint fine-tuning across highly imbalanced modalities?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "There is novelty in this work. Specifically, there is clear architectural separation between domain-specific encoders and a unified fusion backbone which promotes flexibility and domain transfer.\n\nExperiment results are also promising. This work outperforms strong baselines (MolX, ChemBERTa, GraphMVP) on multimodal reasoning tasks, particularly in low-data and cross-domain settings.\n\nFigure 4 is particularly useful as it shows module-wise attribution analyses for how modality-specific knowledge contributes to chemical reasoning.", "weaknesses": "There is however limited novelty at the core LLM level. While modularization is effective, the language model itself is adapted rather than fundamentally redesigned for chemistry.\n\nThere is also lack of validation of the practicality of this approach, say on real world sparse datasets. The evaluation focuses primarily on benchmark datasets, with minimal discussion of noisy experimental spectra or reaction data.\n\nFurther, the computational cost of the work seems infeasible. Training multiple modality-specific experts and fusion layers may hinder accessibility for smaller research groups.\n\nThere are also couple recent relevant research works that have not been referenced which the authors need to cite to improve the comprehensiveness of the related work section:\n\n- Le, Khiem, Zhichun Guo, Kaiwen Dong, Xiaobao Huang, Bozhao Nan, Roshni Iyer, Xiangliang Zhang, Olaf Wiest, Wei Wang, and Nitesh V. Chawla. “MolX: Enhancing Large Language Models for Molecular Understanding With A Multi-Modal Extension.” Proceedings of the 2025 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, AC M, 2025. \n\n- Ju, Jiaxin; Yizhen Zheng; Huan Yee Koh; Shirui Pan. “Uni-MRL: Unified MultiModal Molecular Representation Learning with Large Language Models and Graph Neural Networks.” Advances in Knowledge Discovery and Data Mining (PAKDD 2025), Lecture Notes in Computer Science, vol. 15874, Springer, 2025, pp. 275-287.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CSnBQFRsyU", "reviewer": "Reviewer_XTAL", "summary": "The paper “mCLM: A Modular Chemical Language Model That Generates Functional and Makeable Molecules” introduces mCLM, a new type of chemical language model designed to generate small molecules that are both functionally effective and synthetically feasible. Traditional large language models can understand chemical information, but they often struggle to design molecules that can actually be synthesized in the lab. mCLM addresses this challenge by shifting from atom-level representations (like SMILES strings) to a modular representation, where molecules are described as combinations of chemically meaningful, synthesis-ready building blocks. This allows digital molecular generation to directly correspond to real-world automated synthesis.\n\nmCLM functions as a bilingual model—it understands both natural language descriptions and molecular structures. It uses graph neural networks (GNNs) to encode molecular modules and combines these with text embeddings in a Transformer-based architecture, enabling it to “code-switch” between chemistry and natural language. The model is trained on paired datasets that link chemical properties, synthesis reactions, and textual descriptions. This training allows mCLM not only to generate molecules with desired biological or physical functions but also to ensure that these molecules are makeable through automated synthesis pipelines.\n\nIn experimental evaluations using 430 FDA-approved drugs and 122 out-of-distribution compounds, mCLM demonstrated substantial improvements in key pharmacological properties—including absorption, distribution, metabolism, excretion, and toxicity (ADMET)—while maintaining or improving synthetic accessibility. It outperformed leading AI systems such as GPT-5, Claude 3.5, and Gemini 2.5-Flash in both functional property optimization and synthetic feasibility. Furthermore, mCLM proved capable of “rescuing” failed drug candidates—like Evobrutinib and TNG348—by suggesting minimal structural modifications that reduced toxicity and improved drug-like behavior.", "questions": "see weakness.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Conceptually Innovative but Incremental in Execution\nThe idea of representing molecules through modular, synthesis-ready building blocks rather than atom-level encoding is conceptually novel and offers a creative bridge between digital design and physical synthesis. This modular approach reflects an original perspective on chemical language modeling. However, the implementation mainly extends existing ideas from reaction-aware and retrosynthesis-based models, making the innovation more incremental than transformative.\n\n- Solid Technical Foundation but Limited Validation\nThe paper demonstrates technical competence in integrating graph neural networks with Transformer architectures and applying them to chemical structure–language fusion. The system design is coherent, and the methodology is explained at a reasonable level of detail. However, the experimental quality is weakened by the absence of real-world synthesis or bioassay validation, and the evaluation remains largely computational and self-referential, lowering the overall scientific robustness.\n\n- Generally Well-Written but Overclaims Results\nThe manuscript is clear and logically structured, with well-organized figures and a coherent narrative linking modular chemistry to AI language modeling. Nevertheless, some claims—especially about outperforming large general-purpose models and enabling autonomous molecular discovery—are exaggerated relative to the presented evidence. The lack of sufficient methodological transparency (e.g., ablations, dataset details) also detracts from clarity and reproducibility.\n\n- Potentially High Impact but Not Yet Realized\nIf validated experimentally, mCLM could have significant implications for automated drug discovery and robotic chemistry. The integration of function-aware reasoning and synthesis feasibility into a unified framework is a meaningful direction for the field. Yet, given the limited empirical support and narrow demonstration scope, its real-world significance remains largely aspirational rather than achieved.", "weaknesses": "- Limited Generalization and Chemical Creativity\nThe modular tokenization relies on a fixed library of known reaction building blocks and predefined synthesis rules. While this ensures synthetic feasibility, it severely restricts the model’s ability to explore novel chemical spaces or generate fundamentally new scaffolds beyond existing reaction types. Thus, the model’s creativity is constrained by human-curated chemistry knowledge.\n\n- Lack of Experimental Validation and limited ablation and interpretability analysis.\nThe evaluation is almost entirely computational, based on predicted ADMET properties and synthesis scores rather than actual laboratory synthesis or biological testing. Without experimental confirmation, the model’s claimed functional and pharmacological improvements remain speculative and unverified in practice. It does not clearly isolate the contribution of its modular tokenization, GNN encoder, or reasoning loop. The internal decision-making of mCLM—how it balances function optimization and synthesizability—remains a black box, reducing its scientific transparency and reproducibility.\n\n- Weak Comparative and Analytical Rigor\nThe baselines used (e.g., GPT-5, Claude 3.5, Gemini 2.5) are general-purpose models, making comparisons less meaningful. The paper omits direct evaluation against specialized molecular generative models (like ChemBERTa or retrosynthesis-aware VAEs), and lacks ablation studies to show the unique contribution of each module (tokenizer, GNN, reasoning loop).\n\n- Overstated Multimodal Integration and Transparency Issues\nThe bilingual natural language–molecule framework is conceptually attractive but only superficially demonstrated. The model’s interpretability and internal reasoning are not clearly explained, leaving it as a black box with limited insight into how function and synthesizability are balanced.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z2aRlCyv2I", "reviewer": "Reviewer_81Cp", "summary": "This paper proposes mCLM, a framework designed to generate small molecules that are both functional and synthesizable. The model introduces a multimodal framework that combines a GNN based representation of molecular building blocks with natural language understanding of molecular functions. The tokenization is performed at the building block level rather than atom-level, aiming to align molecular representation with automated synthesis rules.", "questions": "- **Reason for free from function group conflicts**: The authors claim that resulting building blocks are *free from functional group conflicts* (lines 263–264). How is this ensured when building blocks from different tokenizers or different molecules are mixed? Wouldn’t incompatible functional groups potentially lead to synthesis failures?\n- What could be the reason that the proposed mCLM show relatively weaker performance on HIA and PGP despite superior results on others?\n- **Proxy model reliability**: Since experimental results rely heavily on proxy models for property prediction, how accurate and robust are these models across different molecular classes?\n- **Ablation study**: Could the authors include an ablation comparing the full GNN-based encoding with a baseline using only textual (SMILES or SELFIES) representations? This would clarify the contribution of the graph modality.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The construction of an LLM framework that jointly considers synthesizability and functionality represents an important step toward practical and interpretable molecular generation.\n- The integration of GNN representations with natural language embeddings for modular chemical reasoning is technically novel and well-motivated.\n- The figures are clean, well-structured, and enhance the overall readability and understanding of the method.", "weaknesses": "I would consider raising the score if the following weaknesses are resolved.\n- **Comparison to fragment-aware baselines**: While the paper includes comparisons to recent general-purpose and domain-specific molecule LLMs, it omits fragment- or group-aware baselines such as SAFE [1], GROUPSELFIES [2], or Reasyn [3]. Even acknowledging that Reasyn is concurrent, such comparisons (especially against Transformer-based models with other representations, as mCLM itself employs a Transformer backbone) would strengthen the evaluation and effectiveness.\n- **Lack of clarity on reasoning knowledge acquisition**: The process for defining and annotating the molecular functions of building blocks (Figure 2) is insufficiently described. How are functional roles such as “Hinge binder, cell activity promoter” obtained or validated? If a user seeks to optimize a given property, does this require (1) manual annotation of new functions, (2) training of a proxy model, and (3) full re-training of the multimodal LLM? The pipeline for expanding functional knowledge is unclear for me.\n- **Unclear link between function-infused vocabulary and molecular functionality**: Section 2 describes the vocabulary as *function-infused* and *synthesis-friendly*. While the synthesis aspect is well justified, the connection between the decomposed building blocks and their *functional meaning* remains ambiguous, even after reading Section 3.3. It is not evident how these building blocks encode or correlate with molecular functions, since molecular function typically arises from overall structure and context, not from isolated building blocks.\n- **Minor correction**: “thanks to recent” in line 161 is aligned with nothing afterwards.\n\n[1] Noutahi, E., et al. Gotta be SAFE: a new framework for molecular design. Digital Discovery, 3(4), 796-804.\n\n[2] Cheng, A. H., et al. Group SELFIES: a robust fragment-based molecular string representation. Digital Discovery, 2(3), 748-758.\n\n[3] Lee, S., et al. Rethinking Molecule Synthesizability with Chain-of-Reaction. arXiv 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KNkb7oVxrM", "reviewer": "Reviewer_XT6a", "summary": "This paper proposes mCLM, a modular chemical-language model that helps the translation between natural language description and molecular blocks. Different from previous chemical LLMs using SMILES or SELFIES as their inputs, mCLM learns a new molecular language that is more effective in property prediction and better in synthesis. Experimental results show that with 3B parameters, mCLM can achieve much better synthetic accessibility and property scores than LLMs like GPT-5.", "questions": "Please address my concerns in the Weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of proposing a new molecular language and tokenizing molecules into substructures is novel and helps LLMs better focus on the structural patterns of molecules that will affect the molecular properties.\n2. The performance of mCLM is significantly much better than previous baselines. With only 3B parameters, mCLM is also much more efficient than LLMs like GPT-5.\n3. This paper is overall well-written and easy to follow.", "weaknesses": "1. The proposed molecular vocabulary lacks sufficient details regarding its implementation. Furthermore, it is worth investigating whether the decomposition of molecules into these substructures could potentially compromise their structural integrity.\n2. The molecular tokenization in mCLM utilizes GNNs to generate embeddings for the building blocks. However, the study does not explore or compare this approach against more advanced or alternative methods, such as VQ-VAE, which could potentially offer superior performance.\n3. The evaluation of mCLM is confined to the Qwen-3B base model. To assess the scalability and generalizability of the proposed method, it would be beneficial to test it across different models of varying sizes and architectures.\n4.  The experimental section primarily benchmarks against other LLMs. While some of the chemical LLMs used as baselines may be outdated, the evaluation notably omits comparisons with established GNN baselines, which are crucial for a comprehensive assessment.\n5. The proposed molecular property optimization task could be significantly strengthened by incorporating more relevant and diverse datasets into the evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)