Conditional Latent Space Molecular Scaffold Optimization for Accelerated Molecular Design
Authors: Onur Boyar
Hiroyuki Hanada
Ichiro Takeuchi
Venue: Accepted by TMLR ()

Abstract: The rapid discovery of new chemical compounds is essential for advancing global health and developing treatments. While generative models show promise in creating novel molecules, challenges remain in ensuring the real-world applicability of these molecules and finding such molecules efficiently. To address this challenge, we introduce Conditional Latent Space Molecular Scaffold Optimization (CLaSMO), which integrates a Conditional Variational Autoencoder (CVAE) with Latent Space Bayesian Optimization (LSBO) to strategically modify molecules while preserving similarity to the original input, effectively framing the task as constrained optimization. Our LSBO setting improves the sample-efficiency of the molecular optimization, and our modification approach helps us to obtain molecules with higher chances of real-world applicability. CLaSMO explores substructures of molecules in a sample-efficient manner by performing BO in the latent space of a CVAE conditioned on the atomic environment of the molecule to be optimized. Our extensive evaluations across diverse optimization tasks—including rediscovery, docking score, and multi‑property optimization—show that CLaSMO efficiently enhances target properties, delivers remarkable sample-efficiency crucial for resource‑limited applications while considering molecular similarity constraints, achieves state of the art performance, and maintains practical synthetic accessibility. We also provide an open-source web application that enables chemical experts to apply CLaSMO in a Human-in-the-Loop setting.

Published in Transactions on Machine Learning Research (09/2025)
Conditional Latent Space Molecular Scaffold Optimization for Accelerated Molecular Design
Onur Boyar
boggrr.owrr.mggyymm@@mmm..ooom
Nagoya University
Hiroyuki Hanada
honada.hiroguki.te how.lda...roonu....../nmaaaaoao. mag..aa............a............
Nagoya Universiby
Ichiro Takeuchi
takeuchi. fakeeichh.cccmm.n/../fmmmmmooook
Nagoya University
RIKEN
Reviewed on OpenReview: https: httpe://openrevieel
openreview. net/ formm?? id= KhaVo99BBJu
Abstract
The rapid discovery of new chemical compounds is essential for advancing global health and developing treatments. While generative models show promise in creating novel molecules, challenges remain in ensuring the real-world applicability of these molecules and finding such molecules efficiently. To address this challenge, we introduce Conditional Latent Space Molecular Scaffold Optimization (CLaSMO), which integrates Conditional Variational Autoencoder (CVAE) with Latent Space Bayesian Optimization (TSSBO) to strategically modify molecules while preserving similarity to the original input, effectively framing the task as constrained optimization. Our LSBO setting improves the sample-eficiencyy the molecular optimization, and our modification approach helps us to obtain molecules with higher chances of real-world applicability. CLaSMO explores substructures of molecules in sample-efficient manner by performing BO in the latent space of CVAE conditioned on the atomic environment of the molecule to be optimized. Our extensive evaluations across diverse optimization tasks-inchuding rediscovery, docking score, and multi-property opti- mization show that CLaSMO efficiently enhances target properties, delivers remarkable sample-efiicinncy crucial for resource- limited applications while considering molecular simi- larity constraints, achieves state the art performance, and maintains practical synthetic accessibility. We also provide an open-source web application that enables chemical experts to apply CLaSMO in Human-in-the--Loog setting.
1 Introduction
The accelerated discovery of chemical compounds represents a crucial challenge with the potential to revolu- tionize global health, offering new ways to combat diseases and viruses. The ability to efficiently discover and develop new chemical compounds could lead to groundbreaking treatments and therapios, addressing some of the most pressing health issues of our time. As the importance of this field grows, so too does the research focused on finding effective solutions. Over the past few years, artificial intelligence (AI) has emerged powerful tool in this endeavor. The combination of increased computational power and advancements in generative modcling has brought us closer than ever achieving significant breakthroughs in accelerated discovery.
Generative models offer various approaches to exploring and creating new chemical compounds. A common strategy involves training a generative model on a comprehensive database of chemical compounds. Once
The source code of this work is available http:///hhhaaacommmmmmme//e/////

---

Published in Transactions on Machine Learning Research (09/2025)
A
B
Input Scaffold
AQED = -0.10
AQEO +0.03
J
D
C
QED 0.8
Agea +0.002
AQED +0.008
An example of the CLaSMO framework updating scaffold for Quantitative Estimation of Drug- likeness (QED) optimization. CLaSMO identifies optimal regions in the CVAE latent space and selects bonding points on the scaffold. Chemical features from these points are used to guide the generation of substructures, which are then integrated into the scaffold (A, B, C, D) through small, targeted modifications to improve molecular properties.
Figure 1:
trained, the model can generate entirely new compounds GómezzBombarelli et al. 2018 Tripp et al.) 2020: Griffiths & Hernández-Lobato 2020 Boyar & Takeuchi 2024 Grosnit et al. 2021; Boyar et al.) 2024 This approach opens the door to discovering novel molecules that are unlike any currently known, potentially revealing a vast, untapped universe of chemical possibilities. However, while the generation of these novel molecules is exciting, their practical applicability is often constrained. The challenges lic in synthesizing these novel molecules and in the limited understanding of their properties, which makes it difficult. for
domain experts to assess their viability (Lim et al. 2020
To address these challenges, another strategy for designing new molccules focuses on modifying/editing existing compounds using generative models (Bradshaw et al. 2019 Lim el al. 2020 reinforcement learning Goottiptt et al., 2020), genetic algorithms [Jensen] 2019), or from the domain experts themselves by trial and error (Bemis& & Mureko, 1996; Schreiber |000 Welsch et al. 2010). These approaches are more likely to produce synthesizable molecules because the base molecule known to exist and, therefore, can be more tractable for real-world applications. However, even with this strategy, significant obstacle remains: sample- efficiency, i.c., how efficiently method finds promising molecular modification with limited number of molecular property evaluations. Evaluating the properties of chemical compound is time-consuning and costly process, and many existing methods in the literature require numerous trials, making them less practical for accolerated discovery. Consequently, there a critical need for a methodology that can generate larget chemical compounds in a more sample-efficien manner.
In this study, we introduce the Conditional Latent Space Molecular Scaffold Optimization (CLaSMO) method, framework designed to address two critical challenges in chemical compound discovery: real-world applicability and sample-efficiency. CLaSMO combines Conditional Variational Autoencoder (CVAE) (Higgins et al. 2016)) with Latent Space Bayesian Optimization (T.SBO) (Gómez-Bombarelii et al. 2018) to strategically modify input molecules and optimize their chemical properties.
In drug discovery, a common strategy for generating synthesizable molecules is to work with molecular scaffolds key substructures that serve as the foundation for chemical modifications and drug design (Bemis) & Murcko 1996). Building on this approach, our framework integrates small substructures into these scaffolds to improve key molecular properties. For modifications to be both effective and synthesizable, it's crucial that the generative model understands how new substructures bond with the scaffold and enhance its properties. To achieve this, we condition substructure generation on the scaffold's chemical features using a CVAE. Our novel data preparation and training strategy enables the CVAE to generate substructures that align with specific atomic environments. We further optimize this process using LSBO, which efficiently explores both the latent space of the CVAE and the scaffold's chemical features. This allows CLaSMO to effectively select regions for modification and generate substructures that are chemically meaningful. By tracking molecular similarity between the initial scaffold and the optimized molecule, CLaSMO is designed to operate in low-
2

---

Published in Transactions on Machine Learning Research (09/2025)
budget scenarios and costly settings, such wet-lab experiments, where high samplo-cfficinney is crucial. It ensures that modifications are both realistic and applicable to real-world contexts. Ultimately, this approach accelerates the discovery of novel, synthesizable compounds with improved properties. Figure [ illustrates an example of molecular optimization with CLaSMO.
We evaluate our approach on a diverse suite of 20 molecular optimization tasks that span a wide range of objectives, including rediscovery of known compounds, multi-property optimization, and drug-likeness enhancement. These tasks present different challenges, enabling comprehensive assessment of our model's performance across varied molecular design scenarios. To further test robustness, we incorporate experimen- tal settings with varying levels of similarity constraints between input and optimized molecules, reflecting both flexible and constrained optimization regimes. In addition to property optimization, we also assess the synthetic and retrosyuthetic accessibility of the generated molecules. Our results show that CLaSMO consistently improves target properties a sample-efficien manner, demonstrating strong generalizability and effectiveness across these diverse tasks.
1. We propose CLaSMO, a pioncering CVAE and LSBO-based molecule modification algorithm for molecular design. We use a novel data preparation strategy that enables CVAF to learn how sub- structures bond with target molecules, providing tailored generations.
2. We show that CLaSMO improves target properties with sample-effiiinny while keeping the opti- mized molecules structurally similar to the input scaffolds, increasing the likelihood of identifying synthesizable compounds with desirable properties.
3. We open source a web-application, tion of imput molecules via CLaSMO, which allows chemical experts to decide the region to modify in input molecule, enabling Human--nnthhe-Loop optimization settings.
that enables interactive optimiza-
2 Related Works
Molecular design strategies can broadly be divided into two categorics: from-scratcc--geneattio of molecules and modification-based approaches. Both categories have made significant strides in recent years, yet they also face unique challenges, particularly regarding real-world applicability and sample-efficincc.
From-scratcch-gnneaatio approaches focus on creating entirely new molecules by conducting scarch to optimize the target property. A seminal work by GomezzBonbarelli et al. (2018) introduced latent space optimization-basee methodology, using VAE to generate novel compounds by navigating the latent space of molecular representations. LSBO builds on this by efficiently reducing the number of expensive black- box evaluations required for molecular optimization, cnabling the discovery of compounds with desirable properties in a continuous latent space. Since then, numerous studies have further refined and expanded the
LSBO framework, focusing on method development and practical applications (Grosnit et al. 2021 Tripp
ct al.
2020 Maus et al. 2022 Griffiths & Hernández-
2020 Boyar & Takcuchi] 2024 Boyar et al. However, like many other generation-frome scratch methods, such methodologies struggle with real- applicabilty--ie.. the difficulty of synthesizing the generated molecules in real-world settings (Lim)
2024
world
2020). Other generation methods, such as genetic algorithms (Jensen) 2019),, Grammar VAE (Kusner 2017). and Junction Tree (JT) VAE (Jin et al., 2018), aim to improve the chemical validity of generated structures. Recent advancements like GP-MOLFORMER (Ross et al.) (024) utilizes large language model- like approaches for molccular design, but the real-world applicability challenge remains a major limitation across these methodologies.
et al.
ct al.
Modification-base approaches, on the other hand, focus adding substructures to existing molecules scaffolds, often Icading to more synthesizable and interpretable designs. Methods like Scaffold-GGM (Lim let al.] 2020) employ a graph generative model to modily molecular scafTolds, thus improving properties with a higher chance of obtaining syuthesizable molecules. Weller & Rohs (0024) introduces DrugHIVE, a deep hicrarchical variational autocncoder that leverages scaffold modification to generate novel molccular compounds. A concurrent study by Lee al.] (2025) proposes a diffusion model basod approach for both from-scratch and scaffold modification--aat generation problems. There are many other scaffold--bsed
3

---

Published in Transactions on Machine Learning Research (09/2025)
Langevin et al.) 2020 not limited to generative modeling (Schreiber] 2000 Welsch et al. 2010 Miao et al. 2011). Techniques like Bradshaw et al. (2019 's model ensure chemical validity in molecular modifications, and Gottipati et al. (2020)'s ' PGFS model uses reinforcement learning to guide additions to the base molocule. These approaches aim to avoid the low real-world viability problem faced by cenerattionffrom-ccree
optimization methodologies [Li et al. 2019
methods, as they build upon known molecular scafTolds, however, they lack advanced optimization methodologies that take samplo-effiiiecy into account.
CLaSMO combines the strengths of both categories, leveraging LSBO in the latent space of CVAE for im- proved sample-efficiene while focusing scaffold- based modifications. Unlike current based molec- ular design methodologies in the literature, CLaSMO does not generate molecules from scratch but instead optimizes molecular properties by adding substructures to existing scaffolds. This approach mitigates the real-world applicability problem, and increases the chance of obtaining molecules that are both effective and practical for real-world synthesis. To evaluate the sample-effiiincy of CLaSMO and benchmark it against established methodologies, we follow the samplo-cffeiincy benchmark introduccd by Gao et al. (2022). This benchmark provides a suite of challenging molecular property optimization tasks, using variely of oracle functions, and compares the performance of numerous molecular design methods from the literature.
A key group of methods included in this benchmark are genetic algorithm- based approaches, which remain competitive and widely applicable to scaffold optimization. Notable examples include Graph-GA (Jensen 2019), which uses graph-based genetic algorithms in combination with a generative model; Smiles GA (Brown
ee al. 2019), which operates on SMILES string representations (Weininger 1988); and STONED (Nigam et al. 2021p, p, which leverages SELFIES (Krenn et al.) 2020), a robust alternative to SMILES for molecular encoding. These methods have demonstrated strong performance and robustness, and benchmarking against
them is highly recommended, as emphasized in Tripp & Hernandeez-Lobate (2023).
Beyond genetic algorithms, other saffodd-pptinizziio-ccaaaa methodologies have also gained attention. For example, MARS Xie et al.] (2021) employs a Markov Chain Monte Carlo (MCMC) approach with an annealing scheme and adaptive proposals to iteratively edit molecular graph fragments, guided by a graph neural nork. Tt has demonstrated strong performance across various lasks, particularly in multi- objective optimization scenarios where properties such as bioactivity, must be jointly optimized. Additionally, reinforcement learning techniques such MolDQN (Zhou et al. 2019b) have proven effective for property optimization lasks. Recent advances in GFlowNets (Bengio el al. 2021 have also opened new avenues in molecular design. For instance, SynFlowNet (Cretu et al. 2024), a GFlowNettbased framework, supports scaffold optimization and has demonstrated competitive performance.
and synthesizability
We adopt and modify the experimental setting proposed by Gao el al. (2022) in Section 5.2 to better align with the scope of our work namely,mely, molecular optimization low-budget scenarios, where oracle evaluations are limited and sample-efficiency is essential. Within this setting, we demonstrate that CLaSMO consistently outperforms baseline methodologies across diverse set of tasks. By combining a targeted scaffold-modification strategy with efficient search, CLaSMO offers balanced and effective solution to molecular optimization under constrained evaluation budgets.
To assess the synthesizability of the generated molecules, we report both the Synthetic Accessibility score (SA score) Ertl & Schuffenhauer (2099) and the Retrosynthetic Accessibility score (RA score) (Thakaar et al., 2021). The SA score is obtained via estimation of synthetic fcasibility based on the occurrence of molecular Tragments in publicly available databases, serving as fast and interpretable proxy for synthetic feasibility. However, it may underestimate difficulty for molecules with rare unconventional motifs, or overestimate it for simple but impractical structures. To address such limitations, we complement this metric with the RA score, which reflects the predicted success of retrosynthetic route planning based on a classifier trained on reaction network outcomes. By incorporating both metrics, we obtain a more comprehensive assessment: SA scores reflect the structural simplicity and prevalence of molecule in known chemical space, while RA scores provide insight into practical synthesis routes using available chemistry knowledge. This dual evaluation is particularly critical in low-data settings, where optimization strategies must not only find high-performing molecules, but also prioritize those amenable to real-world synthesis.
4

---

Published in Transactions on Machine Learning Research (09/2023)
Whole Molecules
Scaffolds
Figure Examples of scaffold extraction from whole molecules. In both the upper and lower rows, side chains are removed, leaving the core structure of the molecule. These resulting scaffolds act as starting points for novel chemical design.
3 Preliminaries and Problem Setup
In this section, we provide preliminary knowledge on CVAEs, LSBO, and scaffolds. We then discuss the challenges of property optimization and scaffold modifications.
3.1 Conditional Variational Autoencoders (CVAEs)
A VAE Kingma & Welling 2014) consists of encoder fone : Z and decoder fdee 2 - X, where X represents the input space and Z the latent space. CVAFs extend the framework of VAEs by incorporating additional condition vector into the latent variable model, facilitating the controlled generation of new instances. In the CVAE architecture, the encoder go(2|æ, c) maps an input æ and a condition e to a latent representation z. Simultaneoussyy the decoder po(z|z, c) reconstructs æ using both z and c. The training of CVAFs formulated as the minimization of the conditional variational lower bound:
J(0, O: x c) = (E [loggo(<>)z c)] + KT. (go(</az p(=)),
(1)
where KL denotes the Bellbackk-eibbeer divergence. In this model, the prior distribution p(z) over the latent variables is typically assumed to be standard normal distribution, I). This assumption simplifies the learning process by standardizing the latent space, ensuring that the encoder learns a distribution that closely aligns with prior distribution, thus enhancing the generative capability of the decoder conditioned on specific contexts.
3.2 Latent Space Bayesian Optimization (LSBO)
In BO, we start with numerous unlabeled instances (æt and smaller set of labeled instances (æi, We) where input æ; X represents chemical compound, and label y4 E ) R indicates its properties such docking scores. BO seeks to optimize costly black-box function JBB X y/ which corresponds to obtaining the physical properties of chemical compounds through experiments or time-consuming simulations in the context of molecular design problems. The goal is to maximize PBB with minimal evaluations, using typically a Gaussian Process (GP) surrogate, trained using the labeled instances L, to predict the function over X. BO uses the surrogate to select an input æ that may yield values surpassing the current maximum m yı- However, building GP surrogate in high-dimensional spaces like chemical compounds is challenging. LSBO tackles this by employing VAE/CVAE trained the unlabelled instances U to reduce dimensionality, encoding instance in X' to lower dimensional latent space Z. This simplifies surrogate modeling and optimization because Z has lower-dimension than X. During LSBO itcrations, the acquisition function applied to the GP's predictions selects new points in Z to evaluate. The chosen latent variable z; is decoded into a new input x. = (x)) This new instance evaluated by fBB and the results update L and refine the GP model. This cycle repeats until optimal results are achieved or resources are exhausted. In contexts like molecular design, LSBO aims discover chemical compounds with optimal properties by efficiently navigating the reduced latent space.
5

---

Published in Transactions on Machine Learning Research (09/2025)
3.3 Scaffolds
Scalfolds (Bemis & Murcko 1996) are the stable core structures within molecules that serve as the framework for chemical modifications in drug design. Scaffolds retain the essential biological activity of the molecule. They play a crucial rolc in molccular design by providing a foundation for chemical modifications aimod at optimizing properties like QED. Researchers often use scaffolds to systematically explore chemical variations (Schreiber 2000; Welsch et al.) 2010), which can lead to the discovery of new compounds with improved properties.
In this study, we follow the scaffold extraction method from Bemis & Murcko (1996), where non-essential components like side chains are removed, leaving the core structure. This extracted scaffold serves as a starting point for further modifications, allowing efficient exploration of chemical space. By focusing on scaffolds, molecular design becomes more streamlined, increasing the likelihood of identifying novel, synthesizable compounds with the desired biological activity. Examples of whole molecule and scaffold pairs are provided in Fig. 2
3.4 Problem Definition
We denote the scaffold, which the base of the modification, as S, and the modified molccule as S'. Our goal is to efficiently find the modification that maximizes the molecular property P which is evaluated by (S"), while keeping the difference between S and S small. Directly iterating over all possible S' to find the best modification that maximizes BB is impractical, as it involves high complexity and costly evaluations of PBB.
The primary challenge in optimizing molecular scaffolds lies in i) determining the optimal bonding point on the base scaffold S, and ii) sclecting the appropriate substructures added to the bonding point to ensure meaningful improvements in the desired property P. A molecular scaffold S, composed of several atoms Pl: P1.P2.....PP Pk, may have atoms with the remaining capacity to form additional chemical bonds. These atoms serve potential candidates for bonding with newly generated substructures. Therefore, the task involves not only selecting the right substructure but also identifying the most suitable bonding point pa to optimize scalfold properties. This adds complexity, as the need for precise modifications must be balanced with the challenges of high-dimensional search spaces and evaluation costs. Consequently, a more efficient approach is required to explore scaffold modifications effectively while minimizing the number of evaluations.
To address this, the problem can be reframed as an optimization task in a reduced latent space Z, obtained through CVAE. In this space, each point E Z corresponds to potential substructure that can be integrated into the scaffold. By encoding the molecular substructures into this lower-dimensionall space Z E Rd, the search becomes more tractable. The objective is to find the optimal latent representation =" conditioned on the optimal bonding point pt that, together, maximize the desired property P when the generated substructure s' + edoc (2) is added to the scaffold. Lot us denote this modification as s' + gS @ s',);, where the function g() adds substructure to the scafTold at p:. The optimization problem then formulated as:
p arg
max 2E EEB(S)
(B') arg max ZEZ.PIEN(S)
BB (g(S pdec (z).
(2)
where BS) is the set of possible bonding points the scaffold S. In Section 4.2 demonstrate that atomic features at Pt are used as condition vectors to generate new substructures, enabling targeted substructure generation for atom pi-
3.4.1 Controlling Molecular Similarity
Current modification-based methods often fail to account for how changes impact molocular similarity be- (ween the original scaffold S and the updated scaffold S', or any structure in general. Adding substructures typically increases molecular weight, which can hinder real- world applicability, especially when exceeding 500 Daltons, as indicated by Lipinski's Rule of Five (Lipinski et al., [201]) Higher molecular weight compounds are more difficult to synthesize, making them less suitable for molecular design. Additionally, sometimes necessary to ensure that modifications result in only minor adjustments to avoid drastic changes.
6

---

Published in Transactions on Machine Learning Research (09/2025)
A
B
C
Encoder
Decoder
Latent Space
Condition Embeddings
[AtomType: Mybridization
Vailence FormalCharga,
Degree, RingMembership
Figure Illustration of the BRICS algorithm and its intogration with the CVAE. The molccule in A is decomposed into substructures using the BRICS algorithm (B), with specific breaking points highlighted. Atomic environment. features extracted from these points to provide critical information about the bonding environment. Embeddings of these features are used as condition vectors, which are concatenated both at the encoder imput along with the substructures and at the latent space of the CVAE (C), guiding the generation substructures that are compatible with the scaffold.
Thus, a key challenge is to guide the optimization ion process by considering molecular similarity, ensuring that the modified molecules remain structurally close to the original scaffold. Such a framework can increase the likelihood of obtaining synthcsizable compounds by limiting divergence from known molecules. In Section we show that our method effectively solves the optimization problem in Eq. (2) while ensuring molecular similarity between S and S
4 Proposed Method
Our proposed CLaSMO framework comprises two key components: the CVAE and the LSBO algorithm. However, an essential first step in our approach is the data preparation required to train the CVAE. In this section, we will begin by outlining the data preparation process, followed by an explanation of the CVAE and the CLaSMO methodology.
4.1 Data Preparation
Our proposed method requires uniquely tailored dataset because no existing dataset in the literature fully moets the specific needs of our approach. To create this dataset, we developed a BRICS (Breaking Relrosyndhelically Interesting Chemical Substructures) (Degen al. 2008) based approach. BRICS is an algorithm designed to decompose organic molecules into smaller, synthetically feasible substructures by identifying breaking points within molecule's structure based chemical retrosynthetic rules. These breaking points, also known division points, are the connections between subgraphs within the molecule that BRICS identifies. The algorithm systematically breaks down molecule M into & substructures, Fig. demonstrates this procedure. BRICS is particularly well-suited for this task because it ensures that the generated substructures are syuthetically feasible and chemically valid, making it idcal for scaffold-based molecular optimization. Unlike other decomposition methods, BRICS explicitly adheres to retrosynthetic rules, cnsuring compatibility with real-world chemical synthesis.
The division points found by BRICS of particular importance because they serve both the points where the molecule is split into substructures and the potential bonding sites where these substructures can be reattached. Therefore, they provide us valuable information about these substructures in terms of what kind of bonds they can form. This dual role makes the division points a crucial piece of information for our generative model. By preserving the properties and features of atoms at these division points, we capture the chemical context around the atom that dictates how substructures can bond with other parts of molecule. The atomic features we consider are atom type, hybridization, valence, formal charge, degree, and ring membership. Detailed explanations of these features provided in the Appendix A.1
7

---

Published in Transactions on Machine Learning Research (09/2025)
These features, when used condition vectors in CVAE, are essential for guiding the generation of substructures that can successfully bond with the scaffold, as they provide detailed understanding of the chemical environment at each division point, such as their atom types and if the selected atom has the capacity for forming additional bonds. Therefore, our dataset for CVAE training includes the substructures obtained via the BRICS algorithm and six different alomic features of the atom at their breaking points. These substructures are represented using a string-based representation method referred to as SELFIES Krenn et al. (2200). The illustration of the BRICS algorithm, atomic feature extraction points, and their integration with CVAE is provided in Fig.
4.2 Substructure CVAE
Our CVAE consists of encoder, parameterized by fæ and decoder, parameterized by fdee The encoder takes the substructure 8 and the associated condition vector c, and maps this input into latent space, producing a latent representation z. The decoder then takes a point from this latent space, along with the condition vector c, and generates a substructure s' that is conditioned on the given atomic properties. The loss function of the proposed CVAE is defined as:
J( (:s.c) |s ( (po (s) e)] + B
(3)
where (0(2|s, c) is the encoder, pe (s|z, c) is the decoder, and 3 balances the reconstruction and KL diver- gence terms (Higgins et al. 2016). The condition vectors allow the CVAE to learn how different substructures interact with specific atomic environments, building deeper understanding of their bonding behavior. Fig- ure 33 demonstrates the inpul. substructures that CVAE uses in its training, and Fig. 3C visualizes the CVAE architecture along with the condition vectors.
After training, if during the generation phase, we provide the CVAE with condition vector that specifies the alomic environment of the target scalfold, the decoder, using a latent vector along with the condition vector, generates substructures that are tailored to bond effectively with the scaffold. This conditioning mechanism ensures that the generated substructures are not random but specifically designed to fit the scaffold's atomic enviromment, improving the efficiency of scaffold modifications and the likelihood of successful bonding, In the proposed CLaSMO method, we jointly optimize the bonding position ps and the latent vector z, conditioned on the atomic properties of the bonding site in order to optimize both the bonding position and the substructure added to the position.
4.2.1 Condition Vector Embeddings
In Section we detailed our data preparation process and the extraction of six atomic environmental features, resulting in a 6-dimensional condition vector. As will explain in Section 5.] the relatively simple nature of the substructures used in CVAE training allows for much lower latent dimension than the actual conditional vector dimension of 6. To align with this simplicity and ensure efficient representation, we employed an Autoencoder model to generate d-dimensional embeddings of the atomic fcatures. These embeddings preserve the key characteristics within the alomic environments and learn the relationship among distinct atomic features while reducing dimensionality, which helps lower computational complexity and improves the model's ability to generalize. After its training, the encoder of the Autoencoder, fene /oodEEad is used to provide the final condition vector to be inputted to CVAE by encoding the six-dimensional atomic environmental feature vector. This process is visualized in Fig. B
4.3 CLaSMO
In CLaSMO, upon training of the CVAE, we perform targeted search in the latent space Z R2 of the CVAE, where decodings from each point z represent a potential substructure to be added to the scaffold. The challenge is to modily the scaffold S by selecting a substructure and integrating it at an appropriate bonding point pt, optimizing the desired molecular property P. Therefore, the search space Ω we consider in our optimization tasks is defined as the Cartesian product of the latent space and the set of possible bonding points on the scaffold:
Ω R (P1,22,... .2......P. P.))
8

---

Published in Transactions on Machine Learning Research (09/2025)
Within this search space, the goal is to modify the scaffold S to maximize the BB function (S").
However, although our main goal is optimizing the target property, we also aim to keep the similarity between S and S" at a certain level. Therefore, in CLaSMO, we apply a similarity constraint using the Dice Similarity Dice 1945) metric to compare the input scaffold S with the modified molecule S' before evaluating the BB function. In order to measure the similarity between S and S', we use Morgan Fingerprints (Morgan 1965; Rogers & Hahn 2010). The Morgan fingerprint of molecule is represented as a binary vector, where each bit indicates the presence absence of specific substructure within the molecule, making them effective and popular for comparing molecular similarities. Using these, the similarity between S and S' is measured by computing the Dice Similarity of their Morgan fingerprint vectors defined as:
DICE(S, S') M| n M'
Ms| M's
where Ms and M's are the Morgan fingerprint vectors of S and S', respectively. Dice Similarity, ranging from 0 (no overlap) to 1 (identical), helps us track structural changes during optimization.
The final optimization objective, incorporating both the search for optimal substructures and the similarity constraint, is given by:
z", =",r arg max fB (S') (z.P.)En
(4)
subject to: DICE(S,S)) 2
where S' + g(S @ fdhe (s'lz", c)). By this approach, CLaSMO efficiently navigates the latent space, opti- mizing molecular properties while allowing modifications only if DICE(S, 2 т, effectively controlling the degree of divergence from the input scaffold.
For the joint optimization of substructures and bonding positions, we consider a GP model: (z.p) + v/a where z is the latent vector representing a substructure, p the bonding position of the modification, and 5/2 := y-y" reprosents the improvement in the property, with y and y' indicating the properties of molecules 8 and S respectively. To prepare the training datasel DD for the GP, we conducted a random sampling of substructures from random latent vectors &, each paired with randomly selected bonding regions p on the scaffold S, and evaluated the v'a obtained from these additions (c.g., in the case of QED optimization, QED score differences between S and S' are calculated). These triplets of (z, p. v/A) are used in GP training. This setup allows the GP model learn the relationship between the latent space representations, atoms in the input scaffold, and the resulting property changes, guiding the optimization process toward regions of the latent space that are more likely to yield beneficial modifications to the scaffold.
Among many possible choices, we employ the Upper Confidence Bound (UCB) acquisition function to guide the optimization process. To ensure LSBO samples from regions that meet the similarity constraint, we introduced a penalization mechanism. Specifically, we assign a negative improvement value y'a when the condition DICE(S, S*) 2 r not satisfied after sampling from pdec([=",c)) Additionally, we also apply a penalty when substructure cannot bond with the target molecule. We outline our approach in Algorithm 1, and provide details about the selection of hyperparameters within our framework in the Appendix A.3
4.3.1 Kernel Design of CLaSMO
The problem we try to solve requires simultaneous optimization over continuous latent vectors and discrete bonding points. To handle this complexity, we use GP model with covariance function & that accom- modates the mixed input space of continuous and discrete variables. We define separate kernels for these inputs:
Kcont (z, x') = exp (- 2 || 11===211))
z'1p).
(- 5pp.ps lg
keae(p.. = exp =exp
Bajusz al. 2015) discusses that the dice similarity one of the best metrics assess similarity between molecules using Morgan fingerprinls.
9

---

Published in Transactions on Machine Learning Research (09/2025)
where (2, x is RBF kernel, and (pt P2) measures the similarity between atoms in the molecule that are ready for additional bond via Kronecker delta function, measuring equality of atom positions, 4 and l2 are the lengthscale parameters for the continuous and categorical components, respectively. The combined kernel used in CLaSMO is then expressed as:
ECLaSMO = k((2.p)) (z', Pý)) = keont x²) keas PPt.
Algorithm CLaSMO
1: Input: GP training data D, Trained CVAE, Trained Autoencoder encoder JoondEmd for condition em- beddings, Input molecules M, Optimization budget molecule K, Similarity threshold Penalization terms \
2: Fit GP using D
for i = M do
Pick molecule m, obtain its scaffold S < Scaffold(m.), Evaluate larget property value y (BB(m.))
for to K do
7:
Identify available atoms in the scaffolds for bonding, Pj E B(S)
8:
Find = = arg max(s.))) fBB (S")
9:
Create condition vector e for atom p; in molecule S:
10:
Obtain condition embeddings - pene
[CondEma
Generate substructure s* + fric (((""((())
11:
12:
Add substructure s* molecule S, at region pt to obtain S if S; + S then
13:
14:
if DICE(S,, S') > then
15:
Evaluate new property: y' = (S)
16:
Compute improvement V' = (y - 1/) if v'a > 0 then Update S, S S
17:
18:
19:
end if
20:
clsc
21:
Set v's to penalization term X
22:
end if
23:
clse
24:
Set V/A to penalization term A2
25:
end if
26:
Update D DU ([=",,")), Update GP with D
27:
28:
end for
29:
end for
5
Experiments
In this section, we first describe the training procedure for our CVAE model (Section 5.1 We start from our dataset selection procedure, and introduce the details of model architecture. We then evaluate the performance of CLaSMO across 20 molecular property optimization tasks Gao et al. (2022), which span a range of objectives including rediscovery of known compounds, similarity-basel optimization, multi-property optimization, and improvements in drug-likenes,
To assess robustness across different optimization regimes, each task repeated under three levels of moloc- ular similarity constraints between the input and optimized molecules, defined by Dice similarity thresholds
Thii identification is performed finding atoms within the molocule that have additional capacity form bonds. For example, carbon can form up four bonds, and it has only three bonds within the molecule, it will be identified available for bonding.
10

---

Published in Transactions on Machine Learning Research (09/2025)
of 0.50, 0.25, and O. This results in total of 60 optimization settings. Higher thresholds enforce stricter structural similarity, enabling controlled, local edits, while lower thresholds allow for broader exploration of chemical space. We also evaluate the synthetic and retrosyuthetic accessibilities of the generated molecules to assess their feasibility from a chemical synthesis and retrosynthetic planning perspective.
Section 5.2 presents the results of these optimization experiments and the corresponding synthetic and retrosyuthetic accessibility analysis. In Section 5.3 we provide an ablation study to examine the impact of conditional generation. Finally, Section 5.4 discusses the extension of CLaSMO to a human-in-the-loop optimization setting.
5.1
CVAE Training
In this section, we start from the dataset selection procedure and then provide model training details.
5.1.1 Dataset Selection
Our motivation for using BRICS-curated dataset is twofold. First, BRICS decomposition enables inter- pretable fragmentation of molecules by producing chemically meaningful substructures with explicit con- nection points. These points can be directly associated with atomic features such atom type, degree, and hybridization, which allows us to condition the generative model on the chemical context at each at- tachment site. Second, since our overall objective is to develop a generative model that performs small, largeted modifications to existing molecules, it is essential to train on substructures that reflect realistic, low-molecular- weightt chemical fragments. This encourages the model to propose minimal, context-aware odits to molccular scaffolds during inference.
To this end, we evaluated two widely used open-source molecular datasets: QM9 Rudddigkeit al. 2012: Ramakrishnan et al. 2014) and ZINC250K (Gómez-Bombarell) et al 2018). We applied BRICS decompo- sition to both datasots and analyzed the resulting fragment distributions with a focus on their suitability in our setting. As shown in Figure 4 BRICS fragments derived from QM9 are compact and chemically consis- tent, with all fragments under 105 Da and most well below 100 Da. In contrast, fragments from ZINC250K exhibit greater chemical complexity and structural variability, with a substantial portion exceeding 100 Da in molecular weight. Representative examples of fragments from both datasets are also presented in Figure
Based this analysis, we selected QM9 as our primary dataset for training the CVAE model. QM9 contains approximately 130,000 small organic molecules and, when processed via BRICS, yields 18,706 unique substructure pairs along with their atomic environment features. The relatively small size and chemical simplicity of these fragments align well with our modeling objective of generating small, synthetically accessible modifications to cxisting scaffolds. Additionally, the compact nature of QM9 fragments climinates the need for further filtering, making the dataset highly consistent and easy to work with. This makes QM9 particularly suitable for our use case, where the goal is preserve high similarity to the input scaffold while introducing meaningful yet minimal structural improvement
5.1.2 Model Training
Among the 18,706 instances, 80% were used for model training, with the remaining data allocated for testing and validation. We represented the molecules using SELFIES (Krenn et al., 2020). string- based molccular representation, in the form of one-hot encoding matrices. Our CVAE architecture consists of three fully connected layers in the encoder and three GRU layers in the decoder. Given the simplicity of the dataset, we determined that a 2-dimensional latent space was sufficient to achieve over 90% reconstruction accuracy on the test set, where training is conducted using an early stopping condition along with a learning rate scheduler. This lower-dimensional space also enhances the performance of BO, which typically excels in smaller-dimensiional spaces. See Appendix A.3 for further details on the selection of hyperparameters in CLaSMO and in training the CVAE model.
*In Appendix A.2 we explore possible exteusions: one where the CVAE is trained exclusively on BRICS fragments derived from the ZINC250K dataset, and another where trained on mixture of QM9 fragments and sixe-compatible fragments filtered from ZINC25KK.
11

---

Published in Transactions on Machine Learning Research (09/2025)
BRICS Fragments from the ZINC250K Dataset
ZINC250K BRICS Fragments
8000
QM9 BRICS Fragments
Frequency
6000
4000
MW: 165.1
MW: 235.0
MW: 360.0
MW: 414.1
BRICS Fragments from the QM9 Dataset
2000
0
100
200
300
400
Fragment Molecular Weight
MW: 58.0
MW: 69.0
MW: 84.1
MW: 100.1
Figure 4: Distribution of BRICS fragment molecular weights from the QM9 and ZINC250K datasets, along with representative examples from each. BRICS fragments from the QM9 dataset generally have lower molecular weights, reflecting its focus on smaller molecules, while the ZINC250K dataset contributes a broader range of fragment sizes.
Additionally, to maintain this simplicity, as explained in Scction 4.2.1 we generated a 2-dimensional em- bedding for the condition vectors using an Autoencoder model, which is trained with fully connected layers in both the encoder and decoder, achieved 93% reconstruction accuracy for the condition vectors. Prior to CVAE training, we obtained the condition vector embeddings via the Autoencoder model, and used them during CVAE training. As a result, the CVAE was trained with both a 2-dimensional latent space for the substructures and 2-dimensional condition vectors, effectively balancing simplicity and optimization performance. Further discussion this design choice is provided in Appendix A.4
5.2 Molecular Optimization Experiments
To comprehensively evaluate the sample-efficiency of our methodology, we applied CLaSMO to the sample- efficiency benchmark proposed by Gao et al.] (2022), where we use 20 diverse molecular optimization tasks. These tasks span various objectives, including broad range of pharrnacenticllly-eeeeaaa oracle functions, and challenging multi-property optimization (MPO) scenarios (Brown ct al. 2019). Instead of limiting the evaluation to narrowly defined lasks, this broad selection enables more comprehensive assessment of each method's generalizability and adaptability across chemically and functionally diverse scenarios.
Here, sample-efficinncy refers to the efficiency with respect to the number evaluations of the black-box function for the larget optimiization or editing lask. For instance, when evaluating molecules through wet-lab experiments, it is often the case that no more than about 100 evaluations can be conducted. Our proposed method, CLaSMO, is specifically designed to achieve such samplo-cfficient molecular editing. On the other hand, in the mumerical experiments presented in this paper, selected problems where the evaluation of the target black-box functions can be performed at relatively low computational cost, allowing for repeated and comprehensive comparisons between the proposed method and existing methods. Hereafter, we refer to these functions as oracle functions. Experiments were conducted with a fixed budget 100 oracle evaluations per seed across seeds, ensuring consistent comparisons. The choice of 100 oracle iterations (in contrast. the 10,000 itcrations used Gao et al. (0022) reflects our focus on developing a methodology tailored for resource-constrained scenarios, which more representative of real-world applications.
CLaSMO was benchmarked against six widely used baseline methods Methods-Smiles GA Brown et al. 2019 Graph-GA (Jensen 2019), Stoned (Nigam et al. 2021), MolDQN (Zhou et al., 2019a). MARS (Xic et al. 2021), and SynFlowNe Net (Cretu et al. 2024) using the same set of 100 scaffolds for all methods as starting molecules. To assess optimization performance across scaffolds of varying sizes, we randomly sampled input scaffolds from both the QM9 and ZINC250K datasots. Appendix AA.5 provides detailed statistics on their molecular weight and property value distributions, along with visualizations of all 100 scaffolds used in the evaluation. As explained below, different. similarity constraints were applied, and open-source implemen-
12

---

Published in Transactions on Machine Learning Research (09/2025)
tations of all baseline methods were used to evaluate their ability to perform molecular optimization while preserving the given scaffold. We note that some of these baseline methods were not designed for sample- efficient settings, and therefore may not perform well under the problem settings aimed at sample-eficieent molecular editing, which the focus of this study
5.2.1 Oracle Definitions and Grouping
To facilitate clearer comparisons and highlight performance patterns, we group the 20 oracles used in our experimental setting into four task-oriented catogorics. The groups and their associated oracle functions are as follows:
Group 1: Similarity and Rediscovery Tasks This group comprises oracles that cither assess molecular similarity to known drug or larget the rediscovery of a known molecule. The oracles in this group include: alhuterol_simlllritnn similarity, celecozib rediscovery, cestranoo_simdlarree thiothix- ene ene_reddesee mee_redscoover,
Group 2: Multi-Property Optimization (MPO) Tasks These involve optimizing multiple
molecular properties simultancously. The oracles in this group include: osimertinib mmpo perindo- pril mpo, ranolazine mpoo silagliptin mpo, zaleplon mpp.
Group 3: Structural Modification Tasks This group encompasses tasks that evaluate a model's ability to generate structurally diverse molecules while preserving specific chemical features. The hopping lasks challenge models to modily the molecule while aiming to mainlain or enhance biological activity despite structural changes. In contrast, the isomers tasks focus on generating different structural arrangements (isomers) of molecule that share the same molecular formula but differ in connectivity spatial orientation. Unlike similarity- tasks that prioritize closeness to reference molecule, these tasks emphasize structural innovation and diversity, assessing a model's capacity to explore novel chemical spaces while retaining desired properties. The oracles in this group include: deco_hop, scaffold_hop isomers cammers_chhn2222 coomers_chh102202210
Group 4: Activity and Benchmark Tasks This group includes oracles that predict biological activity or are part of broader benchmarking suites for drug- likeneis and diversity. The oracles in this group include: drd2, gsk3b, median1, median2, qed, valsarton smarts.
Further details and descriptions of these oracles are provided in Appendix A.7
5.2.2 Property Optimization Results
In this section, we discuss the performance of CLaSMO and benchmark methodologies under three similarity- constraint regimes. In the high constraint setting (7 = 0.50), the optimized molecule must have Dice similarity of at least 0.50 to the input scaffold. In the moderate constraint setting (r 0.25), similarity of at least 0.25 is required, allowing for more flexibility in modifying the scaffold. In these settings, even if the target property can be substantially improved, the result will not be accepted if the similarity falls below the specified threshold. Lastly, in the no constraint setting (T= = 0), there are no restrictions on similarity, and any substructure additions to the scaffold are allowed"
Optimization with High Constraints (Similarity Threshold 0.50) Figure shows the perfor- mance of the methods when the similarity constraint very strict.
In the Similarity and Rediscovery Tasks, CLaSMO is consistently the top performer, providing the top performance from tasks out of 5.
5 Among the competing approaches, Graph-GA, Smiles-GA, and Stoned lack mechanism to ensure preservation of the given scaffold. We implemented hyperparameter optimization for these genetic algorithm--useed approaches to find the best balance for exploration of the search space and scaffold-preserving capabilities. Please refer Appendix A.6 for further details experimental sctting.
6See also the Therapeutics Data Commons more detail: httpa://tdcommon as/functions/oracleen 7 See Appendix A.8 for a further discussion of the rationale behind the chosen similarity constraint thresholds.
13

---

Published in Transactions on Machine Learning Research (09/2025)
Within the MPO Tasks, CLaSMO and MARS methodologies are the two top performers. MARS, method that is specifically designed for MPO settings, outperforms CLaSMO in two out of 5 MPO objectives, and provides a similar performance in osimertinib mpo task.
For the Structural Tasks, CLaSMO gives the best results in deco hop, scaffold Shop and iso- mers_c7h8n202 while MARS achieves a similar result with CLaSMO in the remaining iso- mers chh10222pffel task.
In the Activity and Benchmark Tasks, CLaSMO records the highest values for drd2, mediant and ged, while losing to MARS in gsk3b and median.. None of the methodologies provided an improvement in valsartan smarls task.
Overall, under high constraints, CLaSMO proves to be the best approach, providing the highest Top-10 average scores in 13 out of 20 property optimization tasks.
Optimization with Moderate Constraints (Similarity Threshold 0.25) Figure 6 summarizes the results under a moderate similarity constraint, which allows for more chemical modification while still preserving significant scaffold fcatures.
In the Similarity and Rediscovery Tasks, CLaSMO maintains its lead, where it achieves the highest Top-10 average scores in 4 out of 5 tasks, losing to MARS only in chiothizene_rediscoooee
Among the MPO Tasks, MARS is the top performer, where it provides the highest Top- 10 average scores in out of 5 tasks. CLaSMO outperforms MARS in only zaleplon mpo task, and performs similarly with MARS in osimertinib_ _mpo task.
In the Structural Tasks, CLaSMO gives the best results in all of the 4 tasks in this group.
In the Activity and Benchmark Tasks, CLaSMO records the highest values for medianl, me- dian2 and qed, while losing to MARS in drd2 and gsk3b. None of the methodologies provided improvement in volsarlan smarls lask.
Thus, even with moderate flexibility, CLaSMO is able to harness the allowed modifications to deliver the best overall results, providing the best results in 13 optimization tasks out of 20.
Optimization with No Constraints (Similarity Threshold T = 0) When no similarity constraints are imposed (see Fig. methods only need to keep the scaffold, and all additions to the scaffold are allowed.
Similarity and Rediscovery Tasks, CLaSMO maintains its lead, where it achieves the highest Top-10 average scores in 4 out of 5 tasks, losing to MARS only in cioothinene_rddiccoeeee
In the MPO Tasks, MARS gets the best results in 3 out of 5 objectives, and CLaSMO finds the best results in zaleplon_mme mmpi task and performs similary as MARS in osimertinin_mpae mpo task.
In the Structural Tasks, CLaSMO gives the best results in all of the 4 tasks in this group.
In the Activity and Benchmark Tasks, CLaSMO records the highest values for medianí and ged, while losing to MARS in drd2, gsk3b and median2. None of the methodologies provided an improvement in malartaan_smarts smarts task.
In summary, even without similarity constraints, CLaSMO effectively explores the chemical space to optimize molecular propertics with high sample-cfficiency. In this setting, MARS and Graph-GA methodologies provides more competitive results compared to constrained oplimization settings. However, CLaSMO still achieves the best performance across 11 out of 20 optimization tasks
8In CLaSMO, the ratio of generated subetructures that successfully bond with the given scaffold metric for assessing optimization performance. Appendix A.9 provides detailed analysis of bonding success rates throughout. the optimisation process.
important
14

---

Published in Transactions on Machine Learning Research (09/2025)
celecoxibrredienee rediscoveryyyyyy
deco_hop
drd2
Top 10 Avg Score
0.42
0.25
0.50
0.19
0.39
0.3
0.57
0.13
0.36
0.08
0.33
0.19
0.55
0.03
Iteration
Iteration
Iteration
Iteration
gsk3b
c7h8n2o2
Isoeers
c9h10n2o2pf2c
median1
0.50
0.80
0.43
0.10
Top 10 Avg Score
0.30
0.60
0.37
0.14
0.20
0.5
0.33
0.13
0.10
100
50 Iteration
100
100
50 Iteration
100
Iteration
Iteration
median2
mestranol similaritty
osimertinib mpo
perindopril mpo
0.15
0.34
Top 10 Avg Scott
0.15
0.31
0.71
0.38
0.14
024
0.53
0.32
50
100
50
100
50
100
50 Iteration
100
Iteration
Iteration
Iteration
qed
ranolazine_mpo
scaffold_hop
sitagliptin _mpo
0.93
0.76
0.40
0.25
Top: 10 Avg Score
0.92
0.59
0.46
0.20
0.90
0.43
0.45
0.15
0.87
0.09
0.42
0.05
Iteration
Iteration
Iteration
Iteration
thlothixene rediscovery troglitazone rediscovery,
valsartan smarts
zalepion mpo
0.24
0.25
0.06
0.43
Top-10 10 Avg Score
022
0.22
0.19
0.34
0.22
0.1
0.03
0.30
-0.06
100
50 Iteration
100
100
50
100
Iteration
Iteration
Iteration
CLaSMO
MARS
Smiles-GA Stoned
SynFlowNet
Graph-GA
MolDQN
Figure 5: Comparison of Top- 10 average values obtained at each iteration for CLaSMO, MARS, Smiles-GA, SynFlowNet, Graph-GA, MolDQN, and Stoned using constraint optimization setting, where Dice Similarity threshold - 0.50. CLaSMO identifies molecules with a higher Top-10 average score after 100 iterations in most of the optimization tasks. Among benehmark methodologics, MARS is competitive, followed by Graph-GA and Stoned.
15

---

Published in Transactions on Machine Learning Research (09/2025)
celecoxibrredieeee rediscoveryy
deco_ _hhp
drd2
Top 10 Avg Score
0.45
0.5
0.50
0.44
0.41
0.3
0.57
0.30
0.32
0.19
0.55
0.01
Iteration
Iteration
Iteration
Iteration
gsk3b
c7h8n2o2
1somers
c9h10n2o2pf2ch
median1
0.50
0.86
0.52
0.19
Top 10 Avg Score
0.33
0.62
0.40
0.15
0.22
0.1
0.34
0.13
0.10
100
50 Iteration
100
100
50 Iteration
100
Iteration
Iteration
median2
mestranol similarity
osimertinib mpo
perindopril mpo
0.15
0.36
Top 10 Avg Scort
0.15
0.32
0.71
0.40
0.14
024
0.53
0.33
50
100
50 Iteration
100
50
100
50 Iteration
100
Iteration
Iteration
qed
ranolazine_mpo
scaffold_hop
sitagliptin_ng
0.94
0.78
0.40
0.61
Top: 10 Avg Score
0.92
0.61
0.46
0.47
0.90
0.44
0.45
0.32
0.88
0.26
0.87
0.09
0.42
0.03
Iteration
Iteration
Iteration
Iteration
thlothixene rediscovery troglitazone rediscovery,
valsartan smarts
zaleplon mpo
0.24
0.25
0.06
0.43
Top 10 Avg Score
0.22
0.19
0.34
0.22
0.16
0.03
0.30
-0.06
100
50 Iteration
100
100
50
100
Iteration
Iteration
Iteration
CLaSMO
MARS
Smiles-GA Stoned
SynFlowNet
Graph-GA
MolDQN
Figure 6: Comparison of Top- 10 average values obtained at each iteration for CLaSMO, MARS, Smiles-GA, SynFlowNet, Graph-GA, MolDQN, and Stoned using constraint optimization setting, where Dice Similarity threshold - 0.25. CLaSMO identifies molecules with a higher Top-10 average score after 100 iterations in most of the optimization tasks. Among benehmark methodologics, MARS is competitive, followed by Graph-GA and Stoned.
16

---

Published in Transactions on Machine Learning Research (09/2025)
albuterol similarity
celecoxib rediscovery
deco hop
drd2
Top Avg Score
0.46
025
0.50
0.51
0.41
023
0.57
0.34
0.37
0.32
100
0.19
0.55
100
0.01
50
Iteration
Iteration
Iteration
Iteration
gsk3b
@oomer__ch882ee
isomers | c9t1cc22201220cccccccccc
median1
0.59
0.84
0.54
0.19
Top. 10 Avg Score
0.35
0.62
0.41
0.15
0.22
0.51
0.13
0.10
0.11
100
50
100
50 Iteration
Iteration
Iteration
Iteration
median2
mestranol similarity simlaityy
osimertinib. mpo
perindopril mmp
0.16
0.37
0.44
Top 10 Avg Score
0.15
0.32
0.71
0.40
0.14
024
0.53
0.33
50
100
50 Iteration
100
0
50 Iteration
100
50 Iteration
100
Iteration
qed
ranolazine mpo
scaffold hop
sitagliptin mpo
Top: 10 Avg Score
0.92
0.61
0.46
0.66
0.90
0.44
0.45
0.45
0.88
0.26
0.23
0.87
0.09
0.42
0.02
Iteration
Iteration
Iteration
Iteration
ghiathixene rediscovery troglitazone rediscovery,
valsartan smarts
zaleplon mpo
0.25
0.5
0.06
0.43
Top: 10 Avg Score
0.23
0.19
0.00
0.34
0.22
0.1
...3
0.30
0.21
0:13
0..0
0.25
100
50 Iteration
100
100
50
100
Iteration
Iteration
Iteration
CLaSMO
MARS
Smiles-GA Stoned
SynFlowNet
Graph-GA
MolDQN
Figure 7: Comparison of Top-10 average values obtained at each iteration for CLaSMO, MARS, Smiles- GA, SynFlowNet, Graph-GA, MolDQN, and Stoned, without any constraint threshold. CLaSMO identifies molecules with a higher Top-10 average score after 100 iterations most of the optimization tasks. Among benchmark methodologies, MARS is highly competitive, followed by Graph-GA and Stoned.
17

---

Published in Transactions on Machine Learning Research (09/2025)
QED: 0.615
QED: 0.640
QED: 0.701
QED:
QED: 0.929
QED: 0.938
QED:
QED:
QED: 0.948
Figure 8: Examples of molecules obtained from different CLaSMO experiments in the QED setting. In both rows, the molecules on the left represent the input scaffolds. Each molecule to the right shows a step of substructure addition along with its resulting QED score. The results demonstrate QED improvements from 0.615 to 0.938 and from 0.827 to 0.948 in these examples.
Across all experimental settings, we observe notable differences in sample-eficiency among the benchmarked methods. MolDQN, a reinforcement learning based approach, and SynFlowNet, a GFlowNet-based approach consistently require substantially more oracle evaluations to achieve comparable optimization levels. These findings are consistent with those reported by (Gao al., 2022), which highlighted the low property opti- mization capabilities of such methods Our results further reinforce that their slower convergence makes them less suitable for resource-constrained sample-cffciint optimization tasks, where low evaluation budget is available. In contrast, genetic Algorithn-bbsed methods, especially Graph-GA, remain competitive and often perform well across a variety of tasks. MARS, on the other hand, also shows strong performance, par- ticularly in multi-property optimization scenarios, and achieves solid results overal However, CLaSMO consistently outperforms all baselines across the majority of tasks and similarity constraint levels. These re- sults highlight CLaSMO's robustness and efficiency across both constrained and unconstrained optimization settings, where sample-efficiency essential due to limited evaluation budgets.
To illustrate how CLaSMO optimizes input scaffolds, Fig. g presents two representative examples from the QED optimization task. In each case, CLaSMO incrementally modifies the input scaffold by adding substructures that are conditionally generated based on the local atomic environment. Each step in the optimization process yields a molecule with an improved QED score, demonstrating the model's ability to progressively refine structure strueture-poopett relationships. The examples show QED improvements from 0.615 to 0.938 and from 0.827 to 0.948, underscoring CLaSMO's ability produce chemically valid and property- enhancing modifications through its conditional generation mechanism and LSBO-guided search.
5.2.3 Synthetic and Retrosynthetic Accessibility Evaluation
To assess the synthesizability of the optimized molecules, we analyzed both their synthetic accessibility (SA) scores and retrosynthetic accessibility (RA) scores.
The SA score estimates how difficult a compound is to synthesize based on structural complexity and fragment contributions. Lower SA scores indicate molecules that are easier to synthesize, with values typically ranging from (very easy) to 10 (very difficult). Scores above are generally considered indicative of challenging synthesis Anede al. (2024).
Nowt,, unlike MolDQN, not included in the benchmark conducted by Cao (2022). However, other Celswdet-basced methodologics evaluated in that study exhibited lower optimization performance compared to alternative approaches.
Further discussion on the performance comparison MARS and CLaSMO is provided Appendix A.10
18

---

Published in Transactions on Machine Learning Research (09/2025)
A) T=
B)
0.50
Input Scaffolds Mean SA 3
Input Scaffolds Mean
5
SA Value
SA Value
3
CLaSMO
MARS
CLaSMO
MARS
Figure 9: Distributions of average SA scores for molecules optimized by CLaSMO and MARS under similarity constraints 7 = (A) and = 0.50 (B). Lower scores correspond to higher synthesizabillty.
Given that CLaSMO and MARS exhibited the strongest performance in property optimization, focused our SA and RA analysis on these two _ethod We compared the average SA scores obtained under the lowest and highest similarity constraints, т E {0, 0.50.. Figure presents the distributions of these scores. The horizontal dashed line represents the average SA score of the input scaffolds, which is 3.00.
Both methods generate molecules that are, on average, more complex than the input scaffolds, as the opti- mizod molccules exhibit higher molecular weight and structural complexity. While such increases can pose challenges for synthesis, our results demonstrate that with targeted, small-scale modifications, good synthe- sizability still be maintained. At T 0, the average SA score is 3.88 for CLaSMO and 3.74 for MARS. At == 0.50, these scores improve slightly to 3.73 and 3.68, respectively. Although MARS consistently produces molecules with slightly better synthetic accessibility, CLaSMO maintains competitive synthesizability while achieving stronger performance in property optimization. As indicated by the distribution of SA scores, the generated molecules consistently score below 5, suggesting that their synthesis is likely to be straightforward (Anede et al. 2024).
To complement this prructurebaased evaluation, we further report the RA score, which offers a model-driven estimate of a compound's syuthesizability based its likelihood of being successfully deconstructed by retrosyuthetic planning algorithms. The RA score is computed using machine learning classifier trained on retrosyuthesis outcomes and ranges from 0 to 1, with higher values indicating a higher predicted probability that a valid synthetic route exists. In our evaluation, we report the percentage of molecules that are classified as synthesizable.
As shown in Figure 10 100% of the imput scaffolds are deemed synthesizable, Amoug the optimized molecules from the experiments with similarity threshold 0.5, 88.0% of those generated by CLaSMO and 90.1% of those from MARS are classified synthesizable. For the case where there is similarity threshold, 0, 79.2% of those generated by CLaSMO and 80.6% of those from MARS are classified as synthesizable. These results support the notion that both methods yield molecules that are not only structurally accessible (as evidenced by favorable SA scores) but also recognized by data-driven retrosynthetic models as viable targets for synthesis.
Together, the SA and RA evaluations provide a robust and complementary view of synthesizability, While SA scores reflect fragment-level and structural complexity, RA scores offer practical assessment from the perspective of route planning, CLaSMO, in particular, achieves compelling balance, delivering high amount of property improvements across many tasks while maintaining strong performance across both accessibility metrics and similarity constraints. These findings reinforce its potential for real-world molecular design scenarios where synthetic feasibility is essential.
HiThere strong correlation between the SA and the similarity to the original molecule. When the similarity to the original molecule is high, the SA score also tends be favorable, the original molecule synthesi..ble. Conversely, when tends to be unfavorable. As result, molecules generated by methods that involve minimal
the similarity low, the SA modifications to the original molecule tend receive good SA scores. Therefore, focus comparing CLaSMO and MARS, both of which achieve high scores in property optimization.
19

---

Published in Transactions on Machine Learning Research (09/2025)
100.0%
100-
90
96.0%
79.2% 80.6%
RA Score (%)
80-
70
Input Scaffold
60
CLaSMO
MARS
40
20
10:
Input Scaffolds 0.5
T-
Figure 10: RA scores for input scaffolds and molecules generated by CLaSMO and MARS. Bars indicate the percentage of molecules predicted to be synthesizable by retrosyuthesis planning model.
5.3 Ablation Study
To assess the individual contributions each component the CLaSMO framework, we conducted ablation studies on (i) the role of T.SBO as the search strategy and (ii) the effect of conditioning on atomic environment features in the generative model. All evaluations were conducted in 2D latent space for consistency and under high similarity constraint (T 0.5).
In the first setting, we use a standard VAE without conditioning, paired with LSBO, denoted as L.SBO+VAR In the second, we use the full CVAE model but replace LSBO with random search (RS), denoted as RS+CVAE. These are compared against the original LSBO+CVAE configuration used in CLaSMO. To evaluate these alternatives, we report the final Top-10 average scores across three representative oracle optimization tasks: mostranol sitrraaatiimmlliitty. QED, and troglitazone_rrdisseeee
As shown in Table 1 LSBO+CVAE achieves the highest performance, highlighting the synergy bet guided search and conditional generation. LSBO+VAE underperforms compared the other methods, indi- cating that lack of conditioning leads to chemically incompatible fragments that are penalized under high similarity constraints. While LSBO provides sample-efficient search, cannol overcome this semantic mis- match. RS+CVAE outperforms VAE in two out of three tasks, demonstrating that conditioning helps narrow the search space to chemically meaningful and similarity-compliant substructures even when sampled randomly. These results further underscores the importance of scaffold-awar conditioning in con- strained optimization settings.
Together, these findings confirm that both components, conditional generation and LSBO, are essential for achieving high performance.
Table 1: QED optimization results from the ablation study. We compare three configurations: LSBO+CVAE (original setting), LSBO+VAE (without conditioning). and RS+CVAE (without guided optimization). Re- sults are reported as Top-10 average QED scores under a similarity constraint of r = 0.5. The full CLaSMO configuration achieves the highest performance.
Oracle
LSBO+CVAE
LSBO+VAB RS+CVAE
mestranol similarity
0.331
0.303
0.308
QED
0.930
0.922
0.897
rroglitaaome_rredsccovvey
rediscovery
0.239
0.216
0.191
5.4 Interactive Optimization
In CLaSMO, the modification region the input scaffold is typically selected during the automated op- timization process. However, the framework also supports an interactive mode, where chemical expert manually selects the region of the molecule modify. In this mode, the expert identifies the specific atom region for modification, rather than relying on CLaSMO's automated selection. Once the region is cho-
20

---

Published in Transactions on Machine Learning Research (09/2025)
sen, the rest of the process remains unchanged- CLLSMO continues to optimize the molecule by generating substructures using the CVAE and refining them through LSBO to improve target properties.
This interactive approach offers several key advantages. It enables the integration of expert knowledge into the optimization process, allowing CLaSMO to operate in a cumman-in-theloowe setting. Experts can leverage their domain-specífic insights to target specific regions they find promising, ensuring that modifications are not only data-driven but also aligned with scientific understanding. Meanwhile, CLaSMO maintains its efficient optimization process, using LSBO enhance molecular properties and preserve sample-officinnyy Detailed user guidelines for this open-source web application are provided in the Appendix A.12
6 Conclusion
In this paper, we introduced CLaSMO, a novel framework that combines CVAE and LSBO for scaffold based molecular optimization. Our approach efficiently explores latent spaces to optimize molecular properties in wide variety of objectives, while maintaining structural similarity with the imput scaffold to improve the chances of real-world viability of optimized molecules. By conditioning substructure generation on the atomic environment of the target region in the input molecule, CLaSMO generates chemically meaningful modifications. The experimental property optimization settings-demoustiee demonstrate that CLaSMO consistently delivers superior performance across diverse settings and optimization objectives of varying complexity. By utilizing a pioneering LSBO-basod molecular editing framework that exhibits superior sample-efficinccy, CLaSMO proves to be highly effective methodology for low- low-budget molecular optimization scenarios. Furthermore, CLaSMO's ability control structural divergence via similarity constraints enables consistent performance across diverse optimization tasks, while also yiclding favorable outcomes in terms of synthetic accessibility. Although this paper focuses scalfold-based modilications, CLaSMO is fully compatible with whole molecules, requiring no changes to its methodology. Additionally, we have open-sourced web application to allow chemical experts to use CLaSMO in a Human-in-the-Loop setting, further extending its practical applicability. Overall, CLaSMO exemplifies the power of combining scaffold- strategics with LSBO, offering a highly effective tool for targeted drug discovery and broader molecular design challenges.
tasks such as rediscovery of molecules and multi-
Limitations and Scope
As discussed throughout this manuscript, CLaSMO is particularly well-suited for low-budget optimization scenarios where sanple-effeiency is critical. It is inherently designed to perform additive modifications on given scaffold, which gradually increases the molecular size and weight with each editing step. By operating on scaffolds, CLaSMO helps limit the extent of molecular growth, cnsuring that the optimized molecules do not become too large compared to their original counterparts, with the help of its constrained optimization capabilities. However, the current framework does not support the removal or replacement of existing substructures, limiting its flexibility in certain optimization settings. CLaSMO may not be ideal in scenarios where maintaining fixed molecular size is crucial, where more flexible transformations such as fragment replacement or crossovers between molecular series are required. In such settings, its utility may be limited without integration into broader pipelines. Besides, with its emphasis on introducing small modifications to the input scaffolds and given the scope of this study, CLaSMO uses a set of small fragments in its training dataset. Depending on the requirements of the problem, training on more complex fragments may be necessary, as discussed in Appendix A.2
Another limitation, related to the discussion above, is that CLaSMO is not suitable for de novo, from-scratch molecule generation. While it is effective at adding substructures to a given molecule, it is not designed to construct molecules from scratch. In principle, CLaSMO could be adapted to grow molecule starting from a single atom or fragment by iteratively generating substructures, but such usage falls outside the intended design and is beyond the scope of this work.
Nonetheless, CLaSMO remains a valuable tool when scaffold-based additions are permissible. It can be effectively combined with other methods presented in this study, serving as a component within modular molecular optimization framework tailored to diverse design constraints.
21

---

Published in Transactions on Machine Learning Research (09/2025)
Broader Impact Statement
The work presented in this paper has the potential to accelerate the discovery of new chemical compounds, which can positively impact various industries, particularly pharmaceuticals and materials science. By improving the efficiency of molecular optimization, CLaSMO could contribute to the development of more effective drugs, especially in regions facing significant public health challenges, such the need for rapid vaccine development. Moreover, CLaSMO's focus real-world applicability increases the chances that the compounds discovered are not just theoretical but can be realistically produced, which is critical for translating scientific innovation into real-world solutions. Moreover, CLaSMO's ability to work in a Human- in-the-Loop setting enables domain experts to directly contribute to the optimization process, enhancing collaboration between artificial intelligence and human expertise.
References
Nada Anede, Mebarka Ouassaf, Kannan R. R. Rengasamy, Shafi Ullah Khan, and Bader Y. Alhatlani. Identi- fication and evaluation of natural compounds as potential inhibitors of ns2b-ns3 zika virus protease: A com- putational approach. Molecular Biolechnology, 2024. ISSN 1559-0305. doi: 101000//112233344440333---
URL https:://doo org/10. 00001000.007//00330000--11377771
D. Bajusz, A. Rácz, and K. Héberger. Why tanimoto index an appropriate choice for fingerprint-basod similarity calculations? Journal of Cheminformatiic, 7:20, 2015. doi: 10.11886s133311-01--0000 URL
https://doi. org/ 10.
G. W. Bemis and M. A. Murcko. The properties of known drugs. 1. molecular frameworks. Journal of Medicinal Chemistry, (9(1))22887 2893, Jul 1996. doi: 00.1021/jm960291
Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neurul Information Processing Systems, 84:27381-27388 2021.
Onur Boyar and Ichiro Takeuchi. Latent Space Bayesian Optimization With Latent Data Augmentation for Enhanced Exploration. Newral Computation, 36(11):2446 2444 2478, 10 2024. ISSN 0899-7667. doi: 10.1162/ neco 01708. URL
www.eii..rgg/1..0 wttp////......2...oo... a 01708
Onur Boyar, Yanheng Gu, Yuji Tanaka, Shunsuke Tonogai, Tomoya Itakura and Ichiro Takeuchi. Crystal-
Isbo: Automated design of de novo crystals with latent space bayesian optimization, 2024. URL https:
//arxiv. //arxiv.org/attt wtw.oorgg////aaeeeeeeeeee 2205. 17881
John Bradshaw, Brooks Paige, Matt J Kusner, Marwin Segler, and José Miguel Hernándeez-Loato. A model search for synthesizable molecules. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché-Buc E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3): -101109666188,,,,-11111111111111 2019.
Miruna Cretu, Charles Harris, Ilia Igashov, Arne Schneuing, Marwin Segler, Bruno Correia, Julien Roy, Emmanuel Bengio, and Pietro Liò. Synflownet: Design of diverse and novel molecules with synthesis constraints. arXiv preprint arXiv:2405.011 11.000.011555 2024.
Harm De Vries, Florian Strub, Jérémic Mary, Hugo Larochelle, Olivier Pietquin, and Aaron C Courville. Modulating early visual processing by language. Advances in neural information processing systems, 30, 2017.
Jörg Dogen, Christof Wegscheid-Gerie Gerlach,, Andrea Zaliani, and Matthias Rarcy. On the art of compiling and using 'drug-like' chemical fragment spaces. ChemMedChem, 3(10):1503-150,, 2008. doi: https: //doi.org/1 10. cmd000//mmm..22000007.
22

---

Published in Transactions on Machine Learning Research (09/2025)
Lee R. Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297-300 1945.
ISSN 00129658, 70. URL http:// jstor .rg/sttable/1992409
Peter Ertl and Ansgar Schuffenhauer Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminformatics, 1(1):8, 2009.
doi: 10.1186 0786/175-2288 1111661178-996-111111 1-8. URL https://doi.org /ttp:///.............111111111// 1186/1758-29456
Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor Coley. Sample efficiency matters: a benchmark for practical molecular optimization. Aduances in neural informalion processing systems, 35:21342-21324 2022.
Eduardo C. Garrido-Merchán and Daniel Hernández-Lobatan Dealing with categorical and integer-valued variables in bayesian optimization with gaussian processes. Neurocomputing, 380:20-35, 2020. ISSN 0925-2312. doi:
http://..ccooo......... URL httpp:: http://wwwwwccieeerreettttt com/ /hkeac///rrtcca///////9933331113
Sai Krishma Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, Sarath Chandar, and Yoshua Ben- gio. Learning to navigate the syuthetically accessible chemical space using reinforcement learning. Hal Daumé IIT and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 3668 3679. PMLR, 13 18 Jul 2020. URL http::ttpppppppeeee hatppss//poocceding..omm mlr. www.r.peessvvile wttess/vv1//ae prress//11/ggoottpaaa202a http:/gaittppt....mmmmm html
Ryan-Rhys Griffihhs and José Miguel Hermández-Lobat.. Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chemical science, 11(2):577-588 586, 2020.
Antoine Grosnit, Rasul Tutumov, Alexandre Max Maraval, Ryan-Rhys Griffiths, Alexander Imani Cowen- Rivers, Lin Yang, Lin Zhu, Wenlong Lyu, Zhitang Chen, Jun Wang, Jan Peters, and Haitham Bou-Ammar. High-dimensional bayesian optimisation with variational autoencoders and deep metric learning. ArXiu, abs/2106.03609, 2021.
Rafael Kömeez-Bombanee Jennifer N. Wei, David Duvenaud, José Miguel HernándezzLobato. Benjamin Sánchez-Lengeling, Dennis Sheberla, Jorge Agunlora-pparragiire,, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik. Aulomatic chemical design using a data-driven continuous representation of molecules. ACS Central Science, 4(2):268 276, 2018. doi: 10.1021 10.1022/accceutsc..70077777
Irina Higgins, Loic Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vac: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, 2016.
Jan H Jensen. A graph-based genetic algorithm and generative model/ monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):3567-3575 3572, 2019.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning, pp. 2323 2332, 2018.
Diederik P. Kingma and Max Welling. Auto-encoding variational baycs In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, 2014.
Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alan Aspuru-Guzik. Self- referencing embedded strings (selfies): A 100% robust molecular string representation. Machine Learning: Science and Technology, 1 1(4):045024, oct 2020. doi: 12.1088/2632- 2133222223///1 aba947.
Matt J Kusner, Brooks Paige, and José Miguel Bernández-Lobato. Grammar variational autoencoder. In International Conference on Machine Learning, pp. 1945-1954, 2017.
Maxime Langevin, Hervé Minoux, Maximilien Levesque, and Mare Bianciotto. Scaffold-constrrinee molec- ular generation. Journal of Chemical Information and Modeling, 60(12):5637-556) -646,, 2020.
23

---

Published in Transactions on Machine Learning Research (09/2025)
Scul Lee, Karsten Kreis, Srimukh Prasad Veccham, Meng Liu, Danny Reidenbach, Yuxing Peng, Sace Gopal Paliwal, Weili Nie, and Arash Vahdat. Genmol: A drug discovery generalist with discrete diffusion. In Forty-second International Conference on Machine Learning, 2025. URL inttp:://openrevier net/
forum KMTTPX
Yibo Li, Jianxing Hu, Yanxing Wang, Jiclong Zhou, Liangren Zhang, and Zhenming Liu. Deepscaffold: a comprehensive tool for scaffokd-based de novo drug discovery using deep learning. Journal of chemical information and modeling, 60(1):77 91, 2019.
Jaechang Lim, Sang- Yeon Hwang, Seokhyun Moon, Seungsu Kim, and Woo Youn Kim. Scaffold-based molec- ular design with a graph gencrative model. Chem. Sci., 11:1153-1164, 2020. doi: CC909/9950040
URL http://dx. doi. org/ (039/C95004403A
C. A. Lipinski, F. Lombardo, B. W. Dominy, and P. J. Feeney. Experimental and computational approaches to ostimate solubility and permcability in drug discovery and development settings. Advanced Drug Delivery Reviews, 46(1-3):3 26, March 2001. doi: 10.1016 (0010000/0069900000000001333
Natalie Maus, Haydn Jones, Juston Moore, Matt J Kusner, John Bradshaw, and Jacob Gardner. Local latent space bayesian optimization over structured inputs. Advonces in Neural Information Processing Systems, 1-334505-34114 2022.
Zhenhuan Miao, Julian Levi, and Zhen Cheng. Protein scaffold-based molecular probes for cancer molecular imaging. Amino Acids, 11(:):1037 1047, 2011. doi: 10. 10017/60772--00--066--99
URL https://doiiorrg/
H. L. Morgan. The generation a unique machine description for chemical structures a technique developed chemical abstracts service. Journal of Chemical Documentation, 522:1107-113, 1965. TSSN 0021-9576.
doi: 10.102//1600177011. URL https:///oi.. org/10. 1021/1160011110111 //00:////.....oo//.000000//11110
AkshatKumar Nigam, Robert Pollice, Mario Krenn, Gabriel dos Passos Gomes, and Alan Aspuru- Guzik. Beyond generative models: superfast. traversal, optimization, novelty, exploration and discovery (sloned) algorithm for molecules using selfies. Chemical science, 12(20):7079-7000 7090, 2021.
Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp, and Anatole von Lilienfeld. Quantum chemistry structures and propertics of 134 kilo molccules. Scientific Data, 1, 08 2014. doi: 10.1038/sdata.2011
David Rogers and Mathew Hahn. Extendod-commectivitty fingerprints. Journal of Chemical Information and Modeling, 50(5):742 254, 2010. ISSN 1549-9596. doi: 10.1021/ci1000001 URL htttp:://boioor//// 1021/c1100050t
Jerret Ross, Brian Belgodere, Samuel C Hoffman, Vijil Chenthamarakshann Youssef Mroueh, and Payel Das. Gp-molformer: foundation model for molccular generation. arXiv preprint Xii:22055.49122 2024.
Lars Ruddigkeit, Rund van Deursen, Lorenz C. Blum, and Jean-Louis Reymond. Enumcration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of Chemical Informalion and Modeling, 52(11)22864-288 2012. doi: 10. 10.1021/ci304455d
PMID: 23088335.
Stuart L. Schreiber. Target-oriented and diversity-oriented organic synthesis in drug discovery. Science 287 (5460):1964-1969, 2000. doi: 10.1126/ /111///ciinnc.... (elecncc..8..5.... 07..5600.194 URL https:://ww.scllte wwwp///wwwwccccceeeeee///.////.. org//oi//bss
10. 10.1125/science. 287.5460. 1964
Amol Thakkar, Veronika Chadimová, Esben Jannik Bjerrum, Ola Engkvist, and Jean-Louis Reymond. Retrosynthetic accessibility score (rascore) rapid machine learned synthesizability classification from ai driven retrosynthetic planning. Chem. Sci., 12:3339 3349, 2021. doi: 10.1039/D0SC0540A.. URL
http://dx..o http:://xx.di...... wtt.://..rgg//11110////00050000
Austin Tripp and José Miguel Hernández- Genetic algorithms are strong baselines for molecule generation. arXiv preprint Xrxi::2310. 09267, 2023.
24

---

Published in Transactions on Machine Learning Research (09/2025)
Austin Tripp, Erik Daxberger, and José Hernández-Lobato. Sample-efficient optimization in the latent space of deep generative models via weighted retraining. In Advances in Neural Informalion Processing Systems. 06 2020.
David Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences, 28(1):31 36, 1988. doi: 10. 10.1011/c1005570000
Jesse A Weller and Remo Rohs. Structure-based drug design with a deep hierarchical generative model. Journal of Chemical Information and Modeling, 64(11)::440-66 2024.
Matthew E Welsch, Scott A Snyder, and Brent R. Stockwell. Privileged scaffolds for library design and drug discovery. Current Opinion in Chemical Biology, 14(3):347 361, 2010. ISSN 1367-5931. doi: https: 10t.:01//.....pp..1.....o
Molecular Diversity.
Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molec- ular sampling for multi-objective drug discovery. In International Conference on Learning Representations, 2021.
Chaochao Yan, Sheng Wang, Jinyu Yang, Tingyang Xu, and Junzhou Huang. Re-balancing variational au- toencoder loss for moleculc sequence generation. In Proceedings of the 11th ACM International Conference on Bioinformalics, Compulational Biology and Health Informatics, pp. 1 7, 2020.
Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientūfic reports, 9(1 ):10522, 2019a.
Zhenpeng Zhou, Steven Kearnes, Li Li, Richard Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific reports, 9(1::1 10, 2019b.
A Appendix
A.1 Definitions of Atomic Features
In Table 2
we provide the definitions of the six atomic features we utilized in our data preparation setting.
Property Atom Type
Description
Specifics the clement (c.g., carbon, oxygen), which determines bonding capabilities and chemical behavior.
Hybridization
Describes the mixing of atomic orbitals, influencing shape, bond angles, and bonding interactions.
Valence
Refers to the number of bonds an atom can form, indicating po- tential for additional bonding. Represents the charge all bonding electrons are shared equally, crucial for reactivity and bonding sites.
Formal Charge
Degree
Denotes the number of directly attached atoms (neighbors), pro- viding insight into the local atomic environment. Indicates if the atom is part of ring structure, impacting sub- structure rigidity, stability, and bonding behavior.
Ring Membership
Table 2: Key atomic properties used in CVAE training in CLaSMO framework.
A.2 Training with BRICS Fragments from the ZINC250K Dataset
Our initial decision use BRICS fragments derived from the QM9 dataset was motivated by our goal of training generative model capable of making small, chemically interpretable modifications to input
25

---

Published in Transactions on Machine Learning Research (09/2025)
scaffolds. However, the CLaSMO framework is not limited to the use of QM9. Depending the task hand, the CVAF can be trained on BRICS fragments from alternative datasets that offer different levels of structural complexity and diversity.
As discussed in Section 5.1.1 BRICS fragments obtained from ZINC250K exhibit a wider range of sizes and chemical diversity compared to those from QM9. In this section, through simple experiments, we investigate the implications of using the ZINC250K dataset within the CLaSMO framework.
A.2.1 Training CVAE with Only ZINC250K Fragments
We first conduct a preliminary study which we train the CVAF model using the complete set of BRICS fragments extracted from ZINC250K, without applying any molecular weight filtering. This results in 35,232 unique BRICS fragments and their corresponding atomic environment features, significantly increasing the chemical diversity and structural complexity available during training.
To assess the effect of this incrcased diversity and fragment size, we trained CVAE models using this 2iNe250K-derrvee dataset with latent dimensions of 2 and 8, keeping the model architecture, hyperpa- rameters, and training procedure consistent with the QM9-based setting. The model with a 2-dimensional latent space achieved a test reconstruction accuracy of 87.1%, while the model with an 8-dimensional latent space achieved 91.2%. Despite the added complexity of the ZINC250K fragments, the 2D model still per- formed reasonably well, though further improvements could likely be achieved with proper hyperparameter tuning.
We then evaluated the property optimization performance of the 2D ZINC250K-trained model in a QED op- timization task and benchmarked it against the original CLaSMO model trained on QM9-derived fragments. We also compared the average molecular weight of the optimized molecules.
As shown in Table g+ the use of de fragments results in the generation of molecules with substantially higher molecular weights-averaggee averaging 473.12 Da-due to the inherently larger size of the training fragments. Under the 7 = 0 similarity setting, the ZINC250K-trained model achieves top-10 average QED of 0.890. This performance surpasses all baseline methods except MARS, which it closely approaches, demonstrating that the model remains effective even when operating with a more structurally complex fragmont vocabulary. In the following subsection (Appendix [A.2.)). we show that using small fragments filtered from the ZINC250K dataset in combination with the QM9 datasel is more suitable for our model, which consistently outperforms the baselines, including MARS.
However, under the more restrictive 0.5 setting, the model's performance drops more noticcably, achiev- ing a QED of only 0.871. This decline can be altributed to the larger substructures generated by the model, which make it difficult to maintain high similarity to the input scaffold. The model struggles to produce small, localized modifications, which are essential under stricter similarity constraints. In contrast, the origi- nal QM9-based CLaSMO setting maintains high performance across both similarity thresholds, highlighting the importance of fragment size in preserving molecular similarity during optimization.
It is also important to note that the current model architecture was originally designed for QM9-sized frag- ments and has not been adapted to handle the increased complexity introduced by the ZINC250K fragments, many of which exceed 400 Da in molecular weight. With a more carefully selected model architecture and appropriate hyperparameter muning-such as latent dimensionality, condition embedding size, and decoder capacity we expect this sptup to yield stronger performance. In the next substructure, we show a more suitable alternative to scale CLaSMO with new fragments.
A.2.2 Training CVAE with a Mixture of ZINC250K and QM9 Fragments
As discussed in Section 5.1.1 the ZINC250K dataset contains some BRICS fragments that are comparable in size to those extracted from QM9. By filtering for such fragments, we curated a subset of 4,706 unique BRICS fragments along with their corresponding atomic environment vectors. This subset enhances the chemical diversity of the training data, while maintaining the small fragment size necessary for fine-grained scaffold editing.
26

---

Published in Transactions on Machine Learning Research (09/2025)
Table 3:
Comparison of average optimized molecular weight (Avg. Mol. Weight) and top-10 average QED scores for models trained on BRICS fragments from different datasets.
Avg. Mol. Weight
QED
QED
CLaSMO Sctting
T= 0
T= 0
T 0.50
ZINC250K Fragments, LD:2
473.12
0.890
0.871
QM9 Fragments, LD:2 (Original Setting)
320.08
0.932
0.930
MARS
313.28
0.896
0.890
Comparison of CLaSMO performance across three tasks QED optimization, troglita- zone rediscovery, and mestranol liiisssiiiittiuiiieaeie different fragment settings and similarity constraint 0. The setting that combines QM9 and ZINC250K fragments demonstrates comparable performance to the original QM9- configuration and outperforms the MARS baseline across all tasks, highlighting CLaSMO's robustness and potential for scaling with diverse fragment sources.
Table 4:
CLaSMO Setting
QED
troglitazone rediscovery mestranol similarity
QM9 + ZINC250K Pragments, LD:2
0.912
0.233
0.342
QM9 Fragments, LD:2 (Original Setting)
0.932
0.243
0.355
MARS
0.896
0.216
0.286
We augmented the QM9-derived dataset with filtered subset of BRICS fragments from ZINC250K and trained a CVAE model using a 2-dimensional latent space, consistent with the original setup. No additional hyperparameter tuning was performed; all settings from the original CLaSMO configuration were reused for both model training and the LSBO procedure.
In the QED optimization task, the mixed-fragmenn model achieved a Top-10 average QED of 0.912 surpass- ing all baseline models and closely approaching the performance of the original CLaSMO setup. The average molecular weight of the generated molecules was 327.04, remaining aligned with the QM9-only setting. We further evaluated this configuration on the troglitazone rediscovery and cinttranol_simllanne benchmarks with no similarity constraint (T = 0). As shown in Table 4. the model outperformed the strongest baseline method, MARS, and performed comparably to the original CLaSMO setting across all metrics, confirming the method's robustness to variation in fragment sources.
These results highlight CLaSMO's strong generalization capabilities, even when exposed to more structurally diverse training data. Notably, the model achieved this level of performance without any additional tuning, suggesting that task-specific dataset-aware hyperparameter optimization could lead to further improve- ments. This demonstrates CLaSMO's potential to scale effectively with alternative datasets composed of small molecular fragments.
A.3 Model and Hyperparameter Selection
In chemical VAE models, it has been established that setting the KL divergence weight , < 1 can improve generative performance Yan et al. (020), and our findings are consistent with this. We experimented with a range of 3 values from 1 to 1 1- selecting models based on their reconstruction accuracy on the training set. Interestingly, we found that even at very low values, the CVAF retained generative capacity. However, as I increasod, we observed a decline in both reconstruction accuracy and the diversity of generated molccules.
In chemical VAE models, it has been established that setting the KL divergence weight 3 << 1 can improve generative performance Yan et al. (2020), and our findings are consistent with this. We experimented with range of B values from I to 1- selecting models based on their reconstruction accuracy on the training set. Interestingly, we found that even at very low 3 values, the CVAE retained its generative capacity. However, as increased, we observed a decline in both reconstruction accuracy and the diversity of generated molccules. To ensure robust training, we employed early stopping in conjunction with a learning rate scheduler. Specifically, we used PyTorch's RedueeLROnPlatean, which reduces the learning rate by a factor of 0.5 if the validation loss does not improve for 5 consecutive epochs. Farly stopping was triggered
27

---

Published in Transactions on Machine Learning Research (09/2025)
if no improvement observed within 10 epochs. The initial learning rate was set to 0.001. Models were evaluated based on their reconstruction performance and generative diversity, leading us to select the model with 3 = 0.000001 as the optimal candidate. Additionally, our CVAE leverages conditional batch normalization De Vrics et al. (2017), which improves the impact of conditioning in the generative process.
For the kernel used in CLaSMO, the lengthscale parameters of both the RBF and categorical kernels are learned by maximizing the log marginal likelihood during GP training. In our mixed-variable optimization setup, the contimuous latent vectors and discrete attachment point (atom id) indices are treated jointly, During acquisition function optimization, the categorical variables (attachment point indices) are temporarily relaxed to continuous values. This allows the use of gradient-based optimization across the full search space. After optimization, the relaxed categorical dimensions are rounded to the nearest valid integer index. This approach enables efficient end-to-end optimization while avoiding the computational cost of enumerating over all discrete options.A similar strategy for handling categorical and integer-valued variables in BO is discussed in prior work by Marrido-Merchar & Hernández-Lobato (2020).
For CLaSMO's penalization terms, we opted for straightforward approach rather than an exhaustive hyperparameter optimization process. We set \ -5 to penalize cases where the similarity constraint DICE(S, S') > was violated. Similarly, we assigned x2 = 7.5 for situations where the generated sub- structure could not be added to the input scaffold. These values were chosen to assign poor scores in cases where the sampled region did not meet the desired criteria, allowing LSBO to learn that the region is sub- optimal. This approach helps guide the optimization process away from unproductive regions and toward more promising areas.
A.4 Condition Vector Embeddings
The simplicity of our dataset allowed us to reduce the latent space to two dimensions without compro- mising reconstruction quality, thereby enhancing LSBO performance. Consequently, the decision to use 2-dimensional embeddings for the condition vectors, rather than the original six-dimensional features, was motivated by this low-dimensional latent space configuration in our CVAE model.
Using a six-dimensional condition vector in a CVAE with a 2-dimensional latent space poses a significant challenge: the decoder may overly rely on the condition vectors during reconstruction, potentially diminishing the latent space's ability to encode critical structural information. This can result in loss of critical structural information within the latent space, a disproportionate amount of information is drawn from the condition vectors. Such an imbalance can degrade the quality of the latent space, negatively affecting the surrogate model used in LSBO and ultimately hindoring the optimization process. Conversely, when 2-dimensional condition vectors are used, the latent vectors must encode more information about the input instances, as the decoder's reliance on the condition vectors is reduced. This shift. encourages the latent space to better capture the underlying structural properties, improving its utility for LSBÓ. In LSBO, latent vectors are utilized to train the surrogate model, which guides the scarch process by predicting target property values within specific regions of the latent. space. Therefore, ensuring that the latent vectors are information-rich is crucial for achieving efficient LSBO performance.
To evaluate this effect, we compared CVAE models trained with 2-dimensional embeddings of the conditional features to those trained with the original 6-dimensional features. For both models, we obtained the 2- dimensional latent vectors of input scaffolds using the encoders of the trained CVAES and calculated the corresponding QED values. We then trained random forest regressors to predict the QED values of the input scaffolds based on the latent vectors from each model. Our results showed that the CVAE with 2-dimensional condition embeddings achieved 5% lower mean squared error in QED predictions, underscoring the benefits of rediuced-dimeeniionnll embeddings in preserving latent space quality and improving LSBO performance.
We acknowledge that the model with 6-dimensional condition vectors converged faster and achieved slightly better reconstruction accuracy during its training compared to model with 2-dimensional condition vec- tor embeddings. However, our primary goal was design the latent space for LSBO, where the latent representation provided by the 2-dimensional embeddings proved more effective.
28

---

Published in Transactions on Machine Learning Research (09/2025)
A.5 Input Scaffold Characteristics
In our study, in order to use scaffolds of different sizes and different chemical varieties, used 100 scaffolds sampled from QM9 and ZINC250 datasets. Namely, we used 37 scaffolds from QM9 and 63 scaffolds from ZINC250K datasets. In Figure II we provide the distribution of molecular weights of the input scaffolds along with the distributions of the 20 properties of these scaffolds that are considered in our molecular optimization experiments. Distributions demonstrate that these scaffolds are fairly diverse in many of these 21 distributions. On the other hand, in Fig. 12 we provide the full list of the visualization of the input scaffolds, which ranges from small scaffolds to bigger ones.
A.6 Safile-Efficiency Benchmark Experimental Setting
In the sample-effciency benchmark experiments adopted from Gao et al. 2002 we reduced the number of optimization iterations from 10,000 to 100 to create a low-budget experimental setting, For each methodology we limited the number of oracle calls to 100.
In the original open-source implementation, Smiles-GA uses a population size of 50, while Graph-GA uses a population size of 120. Given our limited optimization budget of only 100 iterations, these values required mdjustment. Furthermore, Smiles-GA allows for 500 mutations in its default configuration, and Graph- GA employs mutation rate of 6.7%. As report results the cases where the scaffold is preserved, we looked for the balance between scaffold preservation and exploration. For this reason, we conducted a grid search for selecting the best-performing sets of population size and mulation rale for the Graph- GA methodology. Population size selected among 4, 5, 10 and the candidate set mutation rate is 6. 6,7%, 4.5%, 1%, 0.1%, 0.05%, 0.01%, 0.005%. The best results were obtained when the mutation rate equal 4.5% and the population size is equal For SMILES-GA, we experimented with population sizes of 4,5, 10 by setting the number of mutations to 10, and best results were obtained when the population size was set to 5.
In the case of SynFlowNet,, the open-source implementation provided by the authors was not directly appli- cable to scaffold optimization tasks. However, the authors kindly guided us on how to adapt their approach for this setting, and we updated their codcbase accordingly based on their recommendations.
The Stoned methodology, although categorized as genetic algorithm in Gao al. (2022), relies on random rearrangements of atoms and lacks specific hyperparameter control the degree of divergence from the initial scaffold. Consequently, similar to MARS, and MolDQN, we used the default hyperparameters for the experiments using Stoned methodology to optimize the given scaffolds.
A.7 Oracle Definitions
In this section, we provide brief descriptions of the oracle functions used in our experiments. For additional details and references, we refer the reader to the Therapeutics Data Commons website; https://tdcommons. at/fucttiossoracclss//
Group 1: Similarity and Rediscovery Tasks
canuterolssmmilrrity" Measures similarity to albuterol, a bronchodilator.
celecoxib retrribbb_ediiceeerry.... Regenerates celecoxib, an anti-inflammatrry drug.
mostranol_similarti Quantifics similarity to mestranol, a synthetic estrogen.
thiothixene rediscovery:
bhichhixene_reedicooeery: Seeks to recover thiothixene, an antipsychotic.
reglicoeroneeeelcccoooery Rediscovery task for troglitazone, a former diabetes drug.
rediscovery:
Group 2: Multi-Property Optimization (MPO) Tasks
29

---

Published in Transactions on Machine Learning Research (09/2025)
MolecullarWeight
albuterol similarit
celecoxib rediscovery
10-
10
10.0
5.0
4-
2.5
2-
0.0
100
200
300
400
0.1
0.2
0.3
0.00
0.05
0.10
0.15
0.20
deco hop
drd2
gsk3b
50
40-
40
30
20
20
10
10
0
0.000 0.025 0.075 0.100
Isomers c7h8n2o2
Isomers c9h10n2o2pf2cl
median1
60
40
20-
10-
0
0
0.0
0.2
0.05
median2
cestrranl_similariyy
05 osimertinib_mm
10.0
60-
7.5
40-
5.0
2.5
0.05
0.10
0.0
0.0
0.6
perindopril mpo
ranolazine
60
8-
40-
20
4
10.
0
0.00
0.15
scaffold hop
sitag sitagliptin mipo
thiothixene rediscovery
40
20
4-
20
0.50
troglitazone rediscovery
valsartan smarts
zaleplon
Figure 11: Distribution of molecular weights and the property values of the input scaffolds.
30

---

Published in Transactions on Machine Learning Research (09/202))
onoso
on
jai
oduo aroo omo
0g.0
cyme
0000
Oud
froo
8
goro
and
o
cog
0
a.o
D
257 2
0Y-
88
A
ogo
0900
040
0,0
950
0-0000
o0ro
O
0
9-100
000
a
010
-00
x
0,0
0-0
8.00
0
9
80
040
8
00
a
6,0
α
8,0
0-000000
o
gomo
Figure 12: Visulizations of input scaffolds.
31

---

Published in Transactions on Machine Learning Research (09/2025)
osimertinib_mmpe Multi-property optimization for osimertinib, an EGFR inhibitor.
parindopril__pol Focused perindopril, ACE inhibitor.
ranolazine_mm mpp: Related ranolazine, an anti-anginal compound.
sitagliptin_ mpo: MPO task for sitagliptin, an antidiabetic drug.
zaleplon_mpo: Targets propertics associated with zaleplon, a sodative-hypnotic.
Group 3: Structural Modification Tasks
deco hop: Scaffold hopping task with partial structural retention.
scaffold__he Generates novel scaffolds while maintaining core activity.
isomers_cchhae Generates valid isomers with the formula 2yHgN202.
1somers_c9h10n222pp Generates valid isomers of Cy
Group 4: Activity and Benchmark Tasks
drd2: Predicts activity against dopamine receptor D2.
gsk3b: Predicts activity against GSK33, kinase target.
medianl, median2: Benchmark tasks requiring similarity across multiple target molecules.
qed: Quantitative Estimate of Drug-likeness.
valsartan_smmantt Checks for specific SMARTS patterns associated with valsartan.
A.8 Similarity Constraints Evaluation
In our experimental framework, molecular similarity constraints are defined using the Dice similarity between the imput scaffold and the optimized molecule. These constraints are designed reflect varying levels of tolerance for structural modifications, which common in real-world molecular design scenarios. However, it is important to note that there is universally accepted threshold for what constitutes "highly similar" molecule, as similarity interpretations can vary depending on the fingerprinting method and application context.
To determine suitable threshold values, we conducted an empirical analysis similarity trends observed during the iterative modification of molecules. In particular, we examined the relationship between similarity and factors such as the number of added fragments, their molecular weights, and the size of the original scaffold. Based on these observations, constraint, as this value permits limited modifications to smaller input scaffolds while still keeping the optimized molecule relatively close to the input scaffold when larger scaffolds are used. Thresholds of 0.25 and 0 are adopted to represent moderate and no similarity constraints, respectively, allowing for increasing flexibility in structural modifications.
designate Dice similarity threshold of 0.5 the high-similarity
Figure 1 illustrates the evolution of both Dice and Tanimoto similarity values as the input scaffold is sequentially modified through fragment additions. As shown, Dice similarity typically falls below 0.5 after only few additions. This behavior is strongly influenced by the size of the input scaffold: smaller scaffolds exhibit more rapid decline in similarity, each modification introduces relatively larger structural change, In contrast, larger scaffolds can accommodate multiple modifications before crossing the same threshold. As detailed in Appendix AA.5] our dataset contains both small and large scaffolds. This analysis reveals that applying a strict similarity constraint disproportionatel) restricts modifications in smaller scaffolds, whereas more relaxed constraint enables least one meaningful modification across broader range of scaffold sizes. We found that setting the high similarity threshold 0.50 provides a good trade-off: it permits a single modification in the smallest scaffolds while maintaining desirable level of similarity for larger ones.
32

---

Published in Transactions on Machine Learning Research (09/2025)
Dice Similarity: 0.67 Tanimoto Similarity: 0.50
Dice Similarity: 0.59 Tanimoto Similarity: 0.42
Dice Similarity: 0.47 Tanimoto Similarity: 0.31
Figure 13:
Illustaaionn of the change in Dice and Tanimoto similarities as the input scaffold gets modified. After a few additions, dice similarity goes below 0.5.
A.9 Analysis on Bonding Success During LSBO
An important metric for evaluating the effectiveness CLaSMO is the success rate of bonding between the generated substructures and the original scaffold. each optimization run, CLaSMO is executed for 100 iterations, and the computational budget is consumed regardless of whether bonding is successful. There- fore, it is crucial to ensure that ondition-based generation strategy consistently produces substructures capable of forming valid bonds with the caffold--spociallly in cases where the scaffold contains atoms with available bonding capacily.
In Table 5 we present an analysis of bonding success across all 20 oracle optimization tasks when the similarity constraint т is set to 0, averaged over all seeds. We report the proportion of successful bonds formed within the first 10, 25, 50, 75, and all 100 iterations. Our findings show that CLaSMO achieves more than 99% bonding success within the first 10 iterations, and over 98% within the first 25 iterations across all oracle tasks. These results demonstrate the high reliability of the model in generating chemically compatible substructures early in the optimization process.
As outlined in Algorithm 1, the molecule is only updated when a generated substructure improves the target property. As optimization progresses and the target property increases, achieving further improvement becomes more difficult. Consequently, LSBO is pushed to explore more sparse and uncertain regions of the latent space search of viable candidates. This behavior especially evident in the latter half of the optimization process. While the success rate of bonding remains above 80% for all oracle tasks up to iteration 50, it begins to decline thereafter. This drop can be attributed both to increased exploration and to reduction in the number of atoms within the scaffold that still capable of forming additional bonds. As result, succossfully generating and bonding new substructures becomes progressively more challenging in later stages.
A.10 More on MARS and CLaSMO Performances
Across the three similarity constraint scttings-no constraint, moderate (r = 0.25), and high similarity (r = 0.50) CLaSMO consistently demonstrates superior optimization performance, ranking first among 13 methods under both moderate and high similarity conditions, and first among 11 methods in the uncon- strained setting. This highlights its robustness and versatility across a wide range of molecular design scenar- ios. In contrast, MARS exhibits strong performance in multi-property optimization tasks, particularly when similarity constraint is imposed. This suggests that MARS is well-suited for exploring broader chemical
33

---

Published in Transactions on Machine Learning Research (09/202)
Table 5: Bonding success rate of generated substructures during LSBO across 20 oracle optimization tasks. The table reports the percentage of substructures that successfully formed a valid bond with the input scaffold within the first 10, 25, 50, 75, and 100 iterations. The results demonstrate high bonding reliability in early iterations, with a gradual decline the search space becomes more sparse and scaffold bonding capacity diminishes.
Property Name
Iter 10
Iter 25
Iter 50
Iter 75
Iter 100
albuterol similariiy
99.9%
99.6%
94.3%
81.3%
64.8%
celecoxib rediscovery
100%
100%
98.0%
90.5%
74.9%
deco _op drd2
100%
98.6%
80.5%
58.1%
44.1%
100%
99.8%
94.5%
81.2%
66.2%
gsk3b
100%
99.8%
94.6%
81.2%
66.3%
isomers c7h8n2o2
99.9%
99.8%
97.4%
85.0%
67.4%
isomers saanues__chh10222pp211
100%
99.9%
97.4%
87.4%
69.3%
medianl
100%
99.9%
97.5%
89.4%
75.0%
median2
100%
100%
98.4%
90.4%
76.8%
mestranol similarity
100%
100%
96.7%
80.3%
61.4%
osimertinib mpo
99.8%
98.2%
82.8%
61.6%
46.5%
periddoprll_mpt
100%
100%
98.1%
86.9%
69.4%
ged ranolazine _mp
99.4%
98.1%
89.2%
70.8%
53.9%
100%
99.8%
90.3%
66.0%
49.6%
scaffold hop
100%
98.6%
81.2%
58.6%
44.3%
sitagliptin _mp thiothixene. rediscovery
99.9%
99.4%
94.5%
77.7%
59.9%
100%
100%
99.0%
93.4%
80.6%
troglitazone poggitazone_rddiccovvvyy
100%
99.9%
95.9%
84.0%
68.5%
valsartan smarts
100%
100%
100%
99.1%
87.3%
zaleplon_ mpo _mp
100%
99.7%
94.3%
75.2%
57.2%
34

---

Published in Transactions on Machine Learning Research (09/2025)
spaces where structural similarity the starting molecule is less critical. As discussed in Section 5.2.3 the SA and RA scores of the molecules generated by both methods are highly comparable, indicating that each approach is capable of producing molecules with high syuthesizability Taken together, these findings suggest that MARS may be the preferred method for multi-property optimization when similarity constraints can relaxed, whereas CLaSMO better suited for scenarios requiring high sample-effciincyy strict structural similarity, or focused singlo-property optimization.
A.11 More on Synthetic Accessibility Scores
Conducting a fair comparison of synthetic accessibility (SA) scores requires focusing on settings where meaningful optimization has occurred. To this end, we included only those experimental configurations in which at least 30 out of 100 input scaffolds were successfully optimized that is, they resulted in an improvement in the target oracle value. This criterion ensures that the SA score analysis reflects the overall syuthesizability of molecules that were effectively optimized.
A.12 Human-in-the-Loop via Web-Application
In Fig. 14 showcase a sequence of screenshots from our web application, demonstrating the process of molecule optimization. First, the user inputs SMILES Weininger (1988) string of the chemical compound into the designated text field. Once the input is provided, the application automatically computes the molecule's QED value and generates a visual representation. Subsequently, the user selects a region of interest by drawing a rectangle around the area they wish to modify. Upon confirming the selection, the CLaSMO optimization process is initiated, targeting improvements in the selected molecular region. Upon completion, the optimized molccule is displayed, and the process can be continued by using the resulting molecule input for further iterations. By incorporating user input in the region selection, create Human-in wn-the-late Loop optimization workflow.
35

---

Published in Transactions on Machine Learning Research (09/202))
Enter SMILES string:
CISC/N22CCCCC2--NCI10
Select property to optimize:
QED
QED Property Value of Input Molecule is: 0.56
Draw rectangle on the molecule select region. To remove the selection, click the trash bin icon.
Highlighted Molecule
Modilied Molecule Similarity 76.92% Change: +38 58%
Selected Atom Index:
Property Value of Optimized Molecule is 0.69
Optimized Molecule for OptthaatOtimizaation
Figure 14: Screenshots from web application showing the step-by-step process of molecule optimization using CLaSMO in an interactive setting. The process includes inputting SMILES string, visualizing the molecule's QED value, selecting region for modification, and initiating the optimization procedure. In this example, the QED value of the input molecule is improved from 0.56 to 0.69, where the resulting molecule is demonstrated the bottom-right figure.
36


Reviews:
[{"review_id": "QvIwiMOgFn", "reviewer": "Reviewer_vekD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed approach seems very suitable to tackle the problem. I like creating the dataset for the VAE\n- The empirical evaluation shows that the method works very well on most tasks\n- Making the project available for chemists in a simple way is great\n\n\nWeaknesses:\n- The novelty is not super high. To me, it seems like the contribution lies mainly in the dataset creation since the CVAE and the LSBO are not super novel. \n- Using only QM9 derived substructures might be limiting diversity etc. I would like to see how the approach scales when using a larger variety of data\n- Some details are missing regarding the GP: see questions.\n- Ablation study: while I like that the conditioning is ablated, I would like to also see the effect of e.g. the LSBO in total (so vs random search), the Kernel selection, potentially a different fragmentation scheme compared to BRICS etc.\n- Since MARS (a rather old technique) has a strong performance I would like to see an even more detailed comparison: a longer paragraph on the approach in related work, a section (maybe in the appendix) to compare where each method excells and where the other fails so that the user knows what approach to pick etc.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a new approach to tackle molecular scaffold optimization. The authors use a combination of a conditioned VAE and BO in its latent space to find new, additional substructures for specific attachment points. To this end, the authors construct a new dataset of suitable molecular substructures from existing molecules using the BRICS fragmentation method. This ensures that the VAE will see synthetizable useful substructures during training. During inference time, this work uses BIO in the latent space of the VAE to produce suitable substructures to attach to the scaffold.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Questions: \n- What fraction of CVAE generations cannot bond or are chemically invalid?\n- What acquisition function is used for the GP? How is it optimized (discrete and continuous components)?\n\n\nRequested changes: \nAdressing the questions and eliminating the weaknesses 2-5", "broader_impact_concerns": "Broader Impact statement is present.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jid5Cjx4hb", "reviewer": "Reviewer_WWB2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n-  The core idea of operating in a conditional latent space determined by the local atomic environment of the attachment point is an interesting addition to existing molecular optimization methodologies, addressing the challenge of ensuring that a proposed modification is chemically sound and likely to bond successfully.\n- The experimental design emphasizes sample efficiency by focusing on a low-budget (100 oracle calls) regime. This setup accurately reflects the constraints of real-world drug discovery and demonstrates the method's performance in resource-limited settings.\n- The use of 20 different optimization oracles and three different similarity constraint levels constitutes a robust assessment of the method's performance in different application domains.\n- The paper is well-written and easy to follow. The problem setup is well-defined and the methodology is explained clearly and logically.\n\nWeaknesses:\n- The claim of improving \"real-world applicability\" is primarily supported by in-silico Synthetic Accessibility (SA) scores. While CLaSMO seems to strike a balance between property optimization and synthesizability, it would be great to provide some more rationale or additional metrics in support of this claim.\n- The CVAE is trained exclusively on fragments derived from the QM9 dataset, which consists of very small and simple molecules. While this aligns with the goal of small modifications, it may limit the model's ability to discover larger or more complex functional group (e.g. those with chlorine atoms). Is there a reason why the CVAE was not trained on more relevant BRICS fragments from e.g. ZINC or ChEMBL?\n- The optimization is performed over a joint space of continuous latent vectors and discrete attachment points. The paper uses a simple kernel that treats each attachment point as independent (via a Kronecker delta). This approach may not scale well for large scaffolds with many potential attachment points, as it fails to leverage chemical similarity between different attachment points (e.g., two sp2 carbons in similar ring systems). Similarly, the currently used attachment point features are inherently local and might fail to capture broader steric and electronic information.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents CLaSMO, a method that extends existing latent space molecular design approaches to scaffold decoration tasks. The key technical contribution is a sample-efficient Conditional Variational Autoencoder (CVAE) trained to generate small, synthetically-plausible substructures. Crucially, the generation is conditioned on the local atomic environment of a specific attachment point on the scaffold of interest and is then integrated with Latent Space Bayesian Optimization (LSBO) to efficiently search for optimal modifications. The LSBO performs a joint optimization over the substructure's continuous latent space and the discrete set of possible attachment points, aiming to improve a target property while maintaining structural similarity to the original molecule.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- The CVAE is trained on fragments from the QM9 dataset, which consists of small, simple molecules. This choice might bias the generator towards producing smaller, less complex substructures. It would be great to add a brief discussion in the Limitations section about how this data choice might affect the model's ability to discover optimal molecules when larger or more complex functional groups are required.\n- What threshold constitutes a high similarity constraint depends on the context and featurization - could you explain the rationale behind choosing a Dice similarity of 0.5 as the high constraint threshold in this setting?\n- The mixed kernel in Sec. 4.3.1 seems to use a single lengthscale. Could you clarify if separate lengthscales were used for the continuous and discrete components (and how they were chosen) or briefly justifying the current design choice?\n- Eq. 2 appears to be missing parenthesis or a composition operator.\n- The symbol $\\mathcal{L}$ is used for both the CVAE loss (Eq. 1) and the set of labeled data (Sec. 3.2). I think it would improve clarity to use distinct notation for these two concepts.", "broader_impact_concerns": "Potential ethical implications are sufficiently addressed in the broader impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XzoZiP9gqU", "reviewer": "Reviewer_pbL8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The proposed method is interesting and relevant (see S1 – S3). However, it remains unclear whether it is applicable only to relatively simple scenarios (QM9-like molecules), or whether it can scale to more complex cases involving more structurally diverse molecular scaffolds.\n\n### Strengths\n(S1 – Relevance): Optimizing a molecule's property while making minimal structural changes is a highly relevant problem in drug discovery. Solutions addressing this challenge are therefore inherently valuable.\n\n(S2 – Clarity & Manuscript Quality): The manuscript is clearly written and presents a well-defined problem and research objective. The proposed method is described with clarity and is well supported by the introduction, background, and a thoughtfully selected review of related work.\n\n(S3 – Significance): The proposed approach consistently outperforms baseline methods across a diverse set of optimization tasks.\n\n### Weaknesses\n(W1 – Potentially Oversimplified CVAE Training Setup): It is unclear why the QM9 dataset was chosen to train the CVAE, given that it only includes a narrow subset of chemical structures (molecules containing only H, C, N, O, and F). This choice limits molecular diversity and may restrict the broader applicability of the approach. A more chemically diverse dataset — such as a filtered subset of PubChem or ChEMBL — might have been more appropriate. Additionally, applying a small-molecule filter at 100 Daltons seems overly restrictive, considering that molecules up to 500 Daltons are typically considered small. Overall, the method appears to have been developed and evaluated in a constrained region of chemical space, potentially resulting in an overly simplified experimental setup (see also Q1, Q2). At present, it is unclear whether the proposed method can scale to more diverse molecular scaffolds (see W2).\n\n(W2 – Limited Insight into CVAE Latent Space and Scalability): The experiments show that simple molecular structures from QM9 can be reconstructed using a low-dimensional (2D) latent space. However, it remains unclear whether a CVAE can effectively model and disentangle more complex molecular structures. This uncertainty limits confidence in the generalizability of the proposed approach. The authors should either demonstrate that their method can also be applied to more structurally complex molecules, or explicitly state that it is currently restricted to generating QM9-like molecules.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes a method for optimizing the properties of small molecules, conditioned on a fixed molecular scaffold. Specifically, the decoder of a Conditional Variational Autoencoder is employed to sample molecular substructures that can potentially be attached to the scaffold at designated attachment points. The optimization process — i.e., identifying a latent variable and an attachment point that together yield a beneficial substructure — is guided by Bayesian optimization, using Gaussian Processes with an Upper Confidence Bound acquisition function. The method is evaluated on 20 molecular optimization tasks, where fitness is assessed via oracle functions. Across these tasks, the proposed approach consistently outperforms the baseline methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "(Q1 – CVAE Latent Space Capacity):\nThe authors report a test-set reconstruction accuracy above 90%, despite using a CVAE with a two-dimensional latent space. In light of (W1), could this high performance be attributed to the simplicity of the molecules and substructures considered? How diverse and complex are the substructures used for training? It would be valuable to see the CVAE trained on a more chemically diverse subset, such as from ChEMBL, to assess whether the latent space can still support accurate reconstruction across a broader range of molecular structures.\n\n(Q2 – Scaffold Diversity in Optimization):\nI assume the optimization tasks are based solely on QM9-derived scaffolds—is this correct? If so, this significantly limits the experimental relevance, as it leaves open the question of whether the approach generalizes to more structurally diverse or drug-like scaffolds. The authors should clarify and explicitly report the source and diversity of the scaffolds used.\n\n(Q3 – Conditioning on Attachment Points Only):\nThe substructure sampling appears to be conditioned only on the attachment point, not on the scaffold as a whole. Is there a specific rationale behind ignoring the scaffold context? If so, the authors should justify this design choice and discuss it in the context of related work. Intuitively, the scaffold might influence the compatibility and desirability of attached substructures, so this simplification deserves further explanation.\n\n(Q4 – Comparison to Diffusion-Based Methods):\nHow do the authors anticipate diffusion-based models would perform in their benchmark tasks? For example, the recently proposed GenMol model [1] includes a section on goal-directed lead optimization. While [1] can be considered concurrent work and is thus not expected to be included in the experimental section, its relevance to the problem setting justifies its discussion in the manuscript.\n\n### Reference:\n[1] Lee et al., GenMol: A Drug Discovery Generalist with Discrete Diffusion, 2025.", "broader_impact_concerns": "no concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)