Offline Model-based Optimization for Real-World Molecular Discovery
Authors: Dong-Hee Shin
Young-Han Son
Hyun Jung Lee
Deok-Joong Lee
Tae-Eui Kam
Venue: ICML 2025 poster (2025)
Keywords: AI for Science
Offline Model-based Optimization
Molecular Optimization
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: A novel framework that stitches molecules from an offline dataset to fine-tune the generative model for offline multi-objective molecular optimization

Abstract: Molecular discovery has attracted significant attention in scientific fields for its ability to generate novel molecules with desirable properties. Although numerous methods have been developed to tackle this problem, most rely on an online setting that requires repeated online evaluation of candidate molecules using the oracle. However, in real-world molecular discovery, the oracle is often represented by wet-lab experiments, making this online setting impractical due to the significant time and resource demands. To fill this gap, we propose the Molecular Stitching (MolStitch) framework, which utilizes a fixed offline dataset to explore and optimize molecules without the need for repeated oracle evaluations. Specifically, MolStitch leverages existing molecules from the offline dataset to generate novel `stitched molecules' that combine their desirable properties. These stitched molecules are then used as training samples to fine-tune the generative model using preference optimization techniques. Experimental results on various offline multi-objective molecular optimization problems validate the effectiveness of MolStitch. The source code is available online.

Offline Model-based Optimization for Real-World Molecular Discovery
Dong-Hee Shin
Young-Han Son
Hyun Jung Lee
Deok-Joong Lee'
Tae-Eui Kam
Abstract
Offline Model-based Optimization (MBO)
Molecular discovery has attracted significant at- tention in scientific fields for its ability to generate novel molecules with desirable properties. Al- though numerous methods have been developed to tackle this problem, most rely on an online set- ting that requires repeated online evaluation of candidate molecules using the oracle. However, in real-world molecular discovery, the oracle is often represented by wet lab experiments, making this online setting impractical due to the signifi- cant time and resource demands. To fill this gap, we propose the Molecular Stitching (MolStitch) framework, which utilizes a fixed offline dataset explore and optimize molecules without the need for repeated oracle evaluations. Specifically, Mol- Stitch leverages existing molecules from the of- fline dataset to generate novel 'stitched molecules' that combine their desirable properties. These stitched molecules are then used as training sam- ples to fine-tune the generative model using pref- erence optimization techniques. Experimental re- sults on various offline multi-objective molecular optimization problems validate the effectivvness of MolStitch. The source code is available online.
win lose
Preference optimization
Augmented (StitchNet) molecules
NeuralNet
Proxy model
Generative
model
Figure I. Proposed oftline MBO process for molecular discovery.
in real-world molecular discovery, the oracle function typically represented by wet lab experiments, which are essource-intensive and can take weeks even months for evaluation (Payton et al., 2023). This creates significant bottleneck, as in silico models cannot receive real-time eval- uation feedback from wet lab. Instead, these models must wait for the wet lab experiments to complete before further optimization can oceur, leading to substantial delays.
To address these challenges, promising research direction is to enable the optimization and refinement of the in silico generative model without relying online evaluation feed- back. To this end, we propose to explore an offline optimiza- tion setting for real-world molecular discovery. Specifically, offline optimization seeks to fully leverage the information contained within an offline dataset, using it to refine the gen- erative model in the absence of online evaluation feedback. Detailed explanations of an offline setting is in Appendix A.
One of the most promising approaches for solving the offline optimization problem is offline model- based optimization (MBO) (Trabucco et al., 2022). In this approach, a proxy, typically parameterized as a deep neural network fe()), is an
1. Introduction
In recent years, a diverse array of in silico generative models has been developed to tackle molecular discovery (Bilodeau et al., 2022). These computational approaches have demon- strated impressive success across various benchmarks, lead- ing to a growing interest in integrating them into real-world applications such as drug discovery. Despite this success, most existing in silico generative models often operate under an online optimization setting, where numerous candidate molecules are generated iteratively and those molecules are evaluated immediately using the oracle function. However,
trained to approximate the oracle function by fitting offline dataset. Once trained, the proxy acts as a surrogate to guide the optimization of the generative model. For in- stance, gradient ascent (Zinkevich, 2003) can be applied to the generative model's parameters based on the proxy's 's pre- dictions, refining the generative model to produce candidate molecules with increasingly desirable properties.
The offline MBO approach exhibits strong performance by generating synthetic data guided by the proxy. However, sev- eral challenges remain: the vanilla proxy is trained using supervised regression loss, which may struggle to accurately approximate the true values from the oracle function as the problem becomes more complex (Fu & Levine, 2021). This challenge is compounded when the proxy encounters out-of- distribution (OOD) data, leading to significant discrepancies
Equal contribution 'Department of Artificial Intelligence, Ko- rea University, Scoul, Republic of Korea, Correspondence to: Tac-Eul Kam <kamte kcrc
Proceedings of the nnnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Offline Model-based Optimization for Real-World Molecular Discovery
Trajectory stitching
Molecular stitching
Stitch
Property Property 2
Property Property 2
Start
Trajectory
Success
Cut
Cut
Molecule A
Start
Trajectory
Fall
Property
1
Property 2
CHiy Stitched molecule
StitchNet
Start
Stitched trajectory
Success
Molecule B
Figure 2. An illustration of trajectory stitching in reinforcement learning (left) and molecular stitching in molecular discovery (right).
between the true values and the proxy's predictions (Qi et al., 2022). To tackle these issues, recent studies have proposed various strategies to enhance the robustness and accuracy of the proxy such as introducing conservative estimates (Tra- bucco et al., 2021), employing local smoothness (Yu et al., 2021), and adopting ensemble methods (Chen et al., 2023a).
In particular, we develop a rank-based proxy that learns the ranking relationship between two molecules based on desired properties and classifies which molecule more favorable. This transformation simplifies the task for the proxy, thereby enabling it to provide more reliable feedback.
Beyond the offline setting, real-world molecular discovery also requires the optimization of multiple molecular ob- jectives (properties). For example, successful drug must satisfy several criteria-it must be bioactive, safe, synthe- sizable, and more-rather than excelling in just one aspect (Fromer & Coley, 2023). This challenge is referred to as multi multi-objective molecular optimization (MOMO) problem. A prevalent approach to tackling MOMO is scalarization, where multiple objectives are combined into a single objec- tive by assigning weights that reflect their relative impor- tance (Gunantara, 2018). However, in the offline setting, the exact importance of each objective is often unknown, and adjusting weights based immediate feedback is limited (Xue et al., 2024). To tackle this, we incorporate priority sampling using a Dirichlet distribution (Minka, 2000) into our framework. Specifically, instead of manually selecting weights, we employ priority sampling to generate a vari- ety of weight configurations during the molecular stitching process, resulting in diverse set of stitched molecules.
While these advanced offline MBO methods significantly enhance the proxy, they may not fully exploit the valuable information inherent in the offline dataset, as this data is typically used solely for training the proxy. In the field of offline reinforcement learning (RL), researchers have intro- duced trajectory stitching techniques (Li et al., 2024; Kim et al., 2024b) to directly leverage the existing offline data by creating synthetic trajectories through segment combination. As depicted in Figure 2. consider two distinct trajectories in the offline dataset: trajectory A has a strong start but ends at the wrong destination, whereas trajectory B starts poorly yet successfully reaches the goal. By applying trajectory stitch- ing, these trajectories can be merged to form new stitched trajectory that combines the strong start of trajectory A with the successful goal achievement of trajectory B.
In this paper, we propose the Molecular Stitching (Mol- Stitch) framework that tackles the offline molecular op- timization problem. Drawing inspiration from trajectory stitching, our framework involves stitching molecules from the offline dataset. For instance, if molecule A possesses desirable property 1 but lacks property 2, while molecule B has the opposite characteristics, we aim to 'stitch' these molecules together to produce new stitched molecule that exhibits both desirable properties. In other words, our frame- work utilizes the existing molecules in the offline dataset to generate novel stitched molecules. allowing the generative model to learn from these newly synthesized data samples.
The main contributions of our framework are outlined as:
We investigate two critical aspects for real-world molec- ular discovery: offline molecular optimization and mulii- objective molecular optimization (MOMO). To this end, we propose MolStitch, the novel framework specifically designed to solve the offline MOMO problem.
MolStitch includes novel components: StitchNer for uti- lizing the offline dataset to generate "stitched molecules', Rank-based proxy for evaluating molecules, and prefer- ence optimization to fine-tune the generative model.
To leverage stitched molecules as augmented synthetic data, they must be effectively evaluated to provide constructive feedback to the generative model. However, this evaluation process poses a challenge, these molecules are unfamiliar the proxy. To address this, we reformulate the proxy's task from property score regression to pairwise classification.
We introduce priority sampling using a Dirichlet distribu- tion to efficiently generate diverse weight configurations. This allows for effective exploration of trade-offs among multiple objectives in the offline MOMO problem.
2

---

Offline Model-based Optimization for Real-World Molecular Discovery
2. Preliminaries
can be implemented. Specifically, the Gg can be updated by maximizing the expected performance of the generated molecules based on the vanilla proxy's predictions such as:
Multi-objective optimization. Let M denote the space of all possible molecules m, and let fi: f2...., . + R be k real-valued molecular objective functions, each repre- senting molecular property to be optimized. The multi- objective molecular optimization problem can be stated as:
arg max Em-G(() ( (m))-
(6)
However, this approach may face challenges as the complex- ity of the problem inereases. The vanilla proxy might pro- duce unreliable predictions when encountering molecules outside its training data distribution, leading potentially misguided optimization. Moreover, this approach may not fully leverage the valuable insights within the offline dataset.
Maximize mEM
F(m) = (2m),,
[u(m)). Ja(mm). (1)
In this problem, it challenging identify and generate single molecule that simultaneously maximizes all objective functions. This challenge arises because improving one molecular property may lead to the deterioration of other properties due to inherent trade-off's between them (Fromer & Coley, 2023). Thus, the goal of this problem is to generate diverse set of molecules that reside on the Pareto front. Definition 2.1 (Pareto front). The Pareto front, denoted as PF, the set of all Pareto optimal molecules mn in the objective space. Mathematically, it can be expressed as:
3. Method
In this section, we present our MolStitch framework for tack- ling the offline MOMO problem. There are three distinct neural networks in our framework: the generative model, StitchNer, and the proxy model. The generative model is designed to generate molecules in textual formats, such as SMILES (Weininger, 1988). StitchNet takes two parent molecules as input and outputs novel stitched molecule that combines desirable properties from both inputs. The proxy serves surrogate for evaluating molecules by classifying which molecule has more desirable properties.
PF- = (F(m") ") m² E PS),
(2)
where PS is the Pareto set that contains all m², defined as:
PS {m* M Dm E M: F(m) ž F(m")
(3)
A F(m) * F(m*))
3.1. Stage 1: Unsupervised Pre-training for StitchNet
Generative model. Let Gg denote a generative model that produces molecules in an auto-regressive manner. The gen- eration process for a molecule m of total length defined:
Stage Unsuperuised pre-training for StitchNet
StitchNet training
ualid
ITe token
Crossouer operator (rule-based)
Offspring molecule
Ga(m) Go (m² ,m==2 ...,, m²), (4)
Molecule A
Maximum Wkelihood
ZINC dataset
Incalid
1=1
ull token
where m' represents the f-th component (or token) in the sequence that constitutes the molecule m.
Molecule
Stitched molecule
StitchNet
Offline optimization. Let D= {m,, F(m,))) be the offline dataset, where m,, E M represents pre-collected molecule and F(m..) represents the corresponding true molecular objective scores. The goal of this offline molecu- lar optimization is to generate new molecules that potentially outperform the best-known Pareto optimal molecules in D. To explore molecular space heyond the dataset D, a common strategy involves constructing a proxy, fo(-) .M + R. to evaluate molecules. The most direct approach is the vanilla proxy, which approximates the scores of true objective func- tions F.)). The vanilla proxy is trained to minimize the discrepancy between the predicted and true objective scores:
Figure 3. In the first pre-training stage, StitchNet gradually learns chemical grammar by imitating rule-based crossover operator.
In the first stage of our framework. we perform unsupervised pre-training for StitchNet using the publicly available ZINC dataset (Sterling & Irwin, 2015). Specifically, we randomly sample two parent molecules from the ZINC dataset and apply rule-based crossover operator (Jensen, 2019) to gen- erate an offspring molecule, as illustrated in Figure 3. Note that this crossover operator adheres to chemical rules and constraints, ensuring that the resulting offspring molccules are chemically valid and potentially exhibit desirable prop- erties (Kamphausen et al., 2002). We then train StitchNet using maximum likelihood objective to generate stitched molecule that closely resembles the offspring molecules derived from the crossover operator. This pre- pre-training en- courages StitchNet to internalize chemical grammar, thereby enabling it to generate chemically valid stitched molecules.
0" arg min
Σ [a((m)) -F(mm))
(5)
To optimize the generative model Go such that it produces molecules with improved objective scores, the vanilla proxy
3

---

Offline Model-based Optimization for Real-World Molecular Discovery
(a) Stage Training proxy model and StitchNet
Proxy training
StitchNet self superuised training
"Molecule is better Molecule 2
uin
Tragmentation
Molecule 1 (score= 0.7)
Fragment molecule A
Molecule
Offline Dataset
Offline Dataset
Stitched molecule
Original molecule
StitchNet
(score 0.8)
Malecule 2 (score
proxy model
Fragment molecule
Molecule 2
(score 0.8)
Chemical feedback
(b) Stage 3: Offline based optimization Generation of stitched molecules
pre-defined number
Generative model fine-tuning
B
PoPo
Stitched molecule i
Winning molecule
Priority sampling A-Dir(a)
Preference optimization
Offline Dataset
Buffer
Buffer
Stitched molecule j
Molecule B
StitchNet
Stitched
Proxy model
Losing
Generative
molecule
model
Figure 4. Main pipeline the MolStitch framework. (a) Stage 2: The rank-based proxy is trained classify which molecule in a given pair has desirable properties, while StitchNet undergoes supe-supe training with chemical feedhack. (h) Stage 3: StitchNet generates stitched molecules, which are stored in a buffer. Once the buffer is full, the proxy evaluates the pairs and selects the superior molecule. Finally, the generative model is fine-tuned using preference optimization techniques favor winning molecules and disfavor losing ones.
3.2. Stage 2: Training the Proxy model and StitchNet
molccule Mau-- If the molecular similarity (Bender & Glen, 2004) between the original molecule and stitched molecules is above certain threshold sim (Meg maa) Z 8, we then train StitchNet Sy using the following loss function:
Rank-based proxy. In the second stage, we obtain the offline dataset that consists of pre-collected molecules along with their true molecular objective scores. To facilitate the process of offline MBO, require proxy model capable of evaluating each molecule effectively. In this work, rather than training the proxy to approximate the exact objective scores, we train it to classify which molecule in a given pair has better objective scores. Specifically, as shown in Figure 4, we sample pairs of molecules from the offline dataset. Since we have access to the ground truth objective scores for each molecule in this dataset, we can establish ranking between the molecules in each pair. We then train our rank-based proxy fo using a pairwise ranking loss:
Latithh (16)
Σ
Bmarnso (
(-log (T)
mong ED
(9)
+ log Spor( + R R( )
where See refers the pre-trained StitchNet that acts as reference model for maintaining chemical validity. The RR (mari)) represents the reward score, serving chemical feedback derived from the given objective scores of Morig This loss Laicch guides Sp. to generate Msa with desir- able objective scores, while not deviating too far from See-
Cproxy (0)
Σ
- log (fo(m)) (fo(m.))-fommm - fo(ma))) (7)
where or (x)) Ite-s is the sigmoid function, and ' is the set of all valid molecule pairs (m m)) within the offline dataset 'D defined as:
Since we are addressing the offline MOMO problem, we cannot query the oracle directly measure the objective scores of Misir for computing R R(msit). Instead, the given objective scores of morig as a form of chemical feedback to approximate the objective scores of mssi This approximation is reasonable because StitchNet generates Msi by recombining fragment molecules that are derived directly from morig. Moreover, we ensure that mair is suf- ficiently similar morig through the similarity threshold 8. This allows assume that the objective scores of m.ii also similar to the objective scores of Morig, as it is widely acknowledged that structurally similar molecules often ex- hibit similar properties and biological activities (Barbosa Horvath, 2004; Alvesalo et al., 2006). Detailed visualization of this self-super training is presented in Appendix I.
utilize
P (mm mi) Maws my D, F(m.) F(m))) (8)
StitchNet. While the pre-training stage focused on training StitchNet to learn chemical grammar and crossover opera- tion, the focus in this stage is to integrate chemical feedback into StitchNet. To achieve this, we first sample the original molecule marig from the offline dataset D. Subsequently, we use the fragmentation function within the rule-based crossover operator to decompose this original molecule into two smaller fragment molecules. StitchNet is then employed to recombine these fragment molecules into new stitched

---

Offline Model-based Optimization for Real-World Molecular Discovery
3.3. Stage 3: Offline Model-based Optimization
model Go by increasing the log-likelihood of generating the winning molecule and decreasing the log-likee oogliikelhhod of the losing molecule. The loss for the generative model is:
In the third stage of our framework, we address the of- fline MOMO problem by utilizing the trained proxy model and StitchNet. The main goal of this stage is to train the generative model to generate novel molecules that poten- tially surpass the best-known molecule in D. In the con- text of the MOMO problem, the scalarization approach is widely adopted, where a weighted sum of multiple objec- tives combined into a single scalar objective, expressed as F(m) = Σi= (m). Here, & denotes the number of objectives, and X; represents the weight assigned to each objective, reflecting its relative importance or priority. How- ever, in an offline setting, the exact importance is often unknown, making it challenging to select proper weights. In addition, the goal of StitchNet is to combine molecules with different characteristics to generate novel stitched molecules that integrate desirable properties from both inputs. Hence, is vital to provide StitchNet with diverse molecule pairs.
Egen (0) - -B BB[loo G)( log Go + Der (Gol)
(12)
where Gzet represents the pre-trained generative model serv- ing as a reference model. The KI. divergence MRI. encour- ages G not to deviate significantly from Gret, ensuring that it maintains adherence to chemical validity. After formulat- ing the initial loss function for the generative model, can draw an intriguing parallel to preference optimization for language models (Rafailov et al., 2023; Tang et al., 2024). In this analogy, our generative model G. can be thought of as the language model and the favorable molecule my as the preferred response. This conceptual alignment allows us to incorporate various preference optimization techniques into our optimization process. Inspired by Direct Preference Optimization (DPO) (Rafailov et al., 2023), we can refor- mulate the Equation 12 into a DPO-like loss by employing the Bradley-Tery model (Bradley & Terry, 1952) such as:
Priority sampling. To address these challenges, we intro- duce priority sampling using the Dirichlet distribution. This sampling approach generates a diverse set of weight config- urations, allowing StitchNet to work with a wide variety of molecule pairs, each focusing on a different balance among multiple objectives. Our choice of the Dirichlet distribution due to its capability to sample directly from the simplex, naturally providing valid weight combinations that are non- negative and sum to 1. The probability density function of the Dirichlet distribution can be expressed by:
Egen = 1 log a (Blog Go( B log Grer (mi) Gorl(m))))
Go(mi))
(13)
This DPO-like loss integrates the separate KI. divergence into single term by utilizing the sigmoid of log odds ra- tios, simplifying the optimization process. However, despite its effectiivness, DPO is known to be prone to overfitting the preference dataset (Hu et al., 2024a). To address this, Identity Preference Optimization (IPO) (Azar et al., 2024) introduces regularization term that penalizes the model when its confidence in the preference margin becomes ex- cessively high. Building upon the concepts of IPO, we can modify the Equation 13 to adopt an IPO-like loss such as:
p(A1,22....... Av A ~~ Dir(a1, ag. o2....,, ax)),
(10)
where Dir(.) refers the Dirichlet distribution, and de- notes the concentration parameters. As illustrated in Figure 4. we use priority sampling A ~ Dir(a1, 02,.... ax)) to sample molecule pairs from the offline dataset. These sam- pled molecules are then fed into StitchNet, which outputs a novel stitched molecule m. This newly generated stitched molecule is subsequently stored in a buffer B and utilized as training sample for the fine-tuning training process of the generative model. Please refer to Appendix J for detailed visualization and the rationale behind priority sampling.
Lgen( = -E [1og ( (Go(muw) Gre((T)) )
2
Gm( (ie((m))))
2,3
(14)
Using the Equation 14, we fine-tune the generative model, REINVENT (Olivecrona et al., 2017), which is an RL-based model widely used for its robust performance across various molecular optimization tasks. More details of the generative model's loss function are in Appendix H, and the pseudo- code for our MolStitch framework is in Appendix K.
Preference optimization (fine-tuning). Once the buffer B populated with a pre-defined number of stitched molecules, we can proceed to fine-tune the generative model. Specifically, we sample pairs of stitched molecules (m, m,) which molecule in each pair is more favorable such as:
from B and use trained rank-based proxy to determine
4. Experiments
{ (m,, (mg m.), otherwise
( my), if fo(m)) > fo(mg) (11)
4.1. Experimental Design and Results
(Me, m+)
Setup. We conducted two main offline MOMO experiments to evaluate the effectiveness of our MolStitch framework. The first experiment focused on the Molecular Property Op- timization (MPO) task (Gao et al., 2022), while the second
where the winning and losing molecules are denoted as Ma and my, respectively. Then, we can update the generative
5

---

Offline Model-based Optimization for Real-World Molecular Discovery
Table 1. Experimental results on the molecular property optimization task under the full-offline setting, with the hest values in bold.
Molecular objectives
GSK38+JNK3
GSK38+INK3+QED
Method
HV(1)
R2(()
HV(t)
R2(()
HV(1)
R2(4)
REINVENT
0,462=0.133
0.229
0.196=0.083
2.646=00327
0.168:000046
3.969++0.644
AugMem
0.489_0.077
0.84510.148
0.272=
0. 1..5::00003
4.101 1:0.46
GraphGA
0.367=0.090
1.116=20.189
0.212=0.063
2.482=#02240
0.200=+0.070
3.973+40.004
Saturn
0.531=0.087
0.785=0.159
0.008
1.977=0.280
0.28 + 058
3.339++02280
GenetioGFN
0.482=0.073
0.869=0.117
0.309=0.087
1.990=0.36 365
0.237=+0.066 066
3.630++:453
Grad
0.494_0.055
0.857.1..12
0.205 0.045
2.502.1..22
4.176.10.319
COMs
0.479=0.063
0.877110.109
0.205
2.496
0.171 0.0062
4.219.10.628
TOM
0.506=0.070
0.807
0.215
2.380
0. 0.195
4.042:10.529
RoMA
0.492=00091
0.843
0.198=0.05
2.537
0. 0.007
4,207:+0.11
Ensemble Proxy
0.500
0.835
0.009
2.462 0.0000
0.215
3..58
BIB
0.486_00.070
0.87410.120
0,203 0.049
2.503 0..245
0. 0.172.10.22
4.080.10.87
BootGen
0.540
0.741
0.225 0.066
2.452
0.201 ::0.074
4.092.10.560
ICT
0.514=0.049
0.827=001104
0.000
2.429=0.386
0. 0.000000000
4.197:+0.593
Tri-Mentoring
0.510=0.042
0.824100079
0.216=0.071
2.458
0. 0.105
4.067:10.467
RaM
0.492=0.062
0.851:00.117
0,282 0.057
2.268 002281
0.249:000049
3,541 1:0412
MolStitch (Ours)
0.579=0.070
0.698=0.122
0.403=00065
1.649=0.259
0.352
2.953++0.571
addressed the docking score optimization task (Lee et al.,
Table 2. Experimental results the docking score optimization task under the full-offline setting, with the best values in bold.
2023). In the first experiment, we closely followed prior studies (Xie et al., b02 Shin et al., 2024b) and adopted four widely used molecular objectives. The objectives in- clude JNK3 and GSK38, which evaluate inhibition against target proteins associated with Alzheimer's disease, along with QED and SA, which are drug- likeness and synthesiz- ability. For the second experiment, we also closely followed recent work (Guo& & Schwaller, 2024b) and targeted the docking score optimization of five proteins-parpl. fa7, jak2, braf, and 5htlb allongide QED and SA. All exper- iments were conducted under offline setting and repeated with 10 different seeds. Further details are in Appendix L.
Target protein
parp1
jak2
braf
fa7
5ht1b
Method
HV(())
HV((T)
HV(()
HV(())
HV(())
REINVENT
0.515
0.477
0.500
0.414
0.509
Saturn
0.528
0.498
0.523
0.431
0.537
GenetieGFN
0.539
0.476
0.508
0.441
0.523
BootGen
0.544
0.496
0.524
0.436
0.545
RaM
0,542
0.488
0.528
0.424
0.525
MolStitch (Ours)
0,560
0.515
0.554
0.451
0.575
Evaluation metrics. The performance of each method was evaluated using two evaluation metrics: the hypervolume indicator (HV) (Zitzler al., 2003) and the R2 indicator (Brockhoff et al., 2012). The HV quantifies the volume of the space dominated by a set of solutions on the Pareto front, where higher values reflect better performance, In contrast, the R2 assesses the quality of a solution set by mea- suring the projection onto pre-defined reference points, with lower values indicating better performance. More detailed explanations of evaluation metries are in Appendix O. Main results. As shown in Table 1, we present the mean HV and R2 performance along with their standard devia- tions for the MPO task under the full-offline setting. We observed that our MolStitch framework consistently demon- strated superior performance across all scenarios with vary- ing numbers of molecular objectives. This underscores the efficacy of our StitchNet in addressing the offline MOMO problem, as it leverages existing molecules to create novel stitched molecules, which serve as valuable training sam- ples for fine-tuning the generative model. Among the com- peting methods, Saturn and GeneticGFN exhibited strong performance, both of which are recent methods that employ genetic algorithms. BootGen and RaM also demonstrated ef- fectiveness as they are recognized as ofate-of-the-an art offline MBO methods for biological sequence design. Furthermore,
Competing methods. We compared our framework against two main categories of methods: molecular optimization and offline optimization. For molecular optimization, we included REINVENT (Olivecrona et al., 2017), AugMem (Guo& & Schwaller, 2024a), GraphGA (Jensen, 2019), Saturn (Guo& & Schwaller, 2024b), and GeneticGFN (Kim et al., 2024a). For offline optimization, we incorporated various offline MBO methods, including Gradient ascent (Grad) (Zinkevich, 2003), COMs (Trabucco et al., 2021), IOM (Qi et al.. 2022). RoMA (Yu et al., 2021), ), Ensemble Proxy (Trabucco et al., 2022), ICT (Yuan et al., 2023), and Tri- Mentoring (Chen et al., 2023a). We also included BIB (Chen et al., 2023b), BootGen (Kim et al., 2023), and RaM (Tan et al., 2025), which are state-of-theaar models for offline op- timization in biological sequence design. Note that used REINVENT as the backbone generative model for all offline optimization methods, not only because it is one of the most robust models for diverse molecular optimization tasks, but also to ensure fairness, as REINVENT also serves as the main backbone model in our framework, For instance, the Grad refers to REINVENT with a vanilla proxy, followed by fine-tuning via gradient ascent. Detailed descriptions of each competing method are presented in Appendix N.
6

---

Offline Model-based Optimization for Real-World Molecular Discovery
Table 3. An ablation study for each component in MolStitch: Rank- based Proxy (RP). StitchNet (SN), and Priority Sampling (PS).
Table 4. Performance comparison of different data augmentation techniques in offline multi multi-objective molecular optimization.
Ablation study
GSK39-JNK3+QED-SAA
Molecular objectives
GSK38+INK3+QED4SA
RP
SN
PS
HV(t)
R2(.)
Data augmentation
HV(1)
R2(1)
0.171 200026
4.176:10.19
Baseline (REINVENT)
0.168=0.046
3.969=00664
0.103
4.132210.502
Stochastic sampling
0.251=00084
3.504:20.334
v v
0.220=000054
3.835++00483
Crossover operator
0.302=0.072
3,110=00479
0.251 1 1.0.844
3.504.10.634
0.289
3.317++0.713
StitchNet (Ours)
0.352.10.000
1.953
0.352:100000
2.953:+0.571
we evaluated the performance of our framework on an addi- tional protein docking score optimization task. As presented in Table MolStitch consistently outperformed all com- peting methods across all five proteins in terms of the HV performance, highlighting the robustness and effectiveness of our framework. Full HV and R2 results for docking score optimization tasks are provided in Appendix C.I and C.2.
training samples for fine- teneettnigg the generative model. Im- portantly, SN incorporates a crossover mechanism similar to that in genetic algorithms but with the added capability of receiving chemical feedback. The efficacy of this crossover operation was validated in our main results, where genetic algorithm-base bbased methods like GeneticGFN and Saturn also demonstrated strong performance. These findings suggest that incorporating the crossover operation, as SN does, is beneficial because it naturally promotes diversity by explor- ing novel combinations derived from existing molecules. Further investigations of SN can be found in Appendix P.
Additional results. Recently, semi-offline optimization, also known as batch hybrid learning, has gained significant attention in the field of large language models (Xiong al., 2024). Specifically, this semi-ofiline setting allows for limited number of online human feedback cycles and en- ables the model to be fine-tuned on new data through large batches. Inspired by this approach, we conducted additional experiments for the semi-offline setting, starting with an offline dataset and periodically querying oracle functions to evaluate molecules in large batches. Due to page limit, the results are deferred Appendix C.4, where our MolStitch framework maintained its superior performance even under the semi-offling setting. We further investigated the impact of using different backbone generative models in MolStitch beyond REINVENT and confirmed its robustness across various backbone models, as detailed Appendix C.5.
Benefits of priority sampling. PS played crucial role in generating diverse weight configurations, which enabled SN to operate with a wide variety of molecule pairs. In the ablation study, PS significantly improved performance by enabling our framework to effectively navigate com- plex Pareto fronts and explore a broader range of trade-offs through diverse weight configurations. Additional insights into the impact of PS, especially in addressing the reward hacking problem in offline MOMO, are in Appendix Q.
4.3. Experimental Analysis and Discussion
Data augmentation. In our main results, we observed that employing StitchNet as data augmentation approach signif- icantly enhanced performance in offline MOMO. To inves- tigate its effectiiveness, we compared StitchNet with other data augmentation techniques. One technique is stochastic sampling, where new molecules are stochastically drawn from the generative model's 's learned distribution. To put it simply, this process can be represented in code-level terms as model. sample () Another technique is the crossover operator, used in GeneticGFN and Saturn, which generates new offspring molecules by combining features from parent molecules in a rule-based manner. As shown in Table 4, all data augmentation techniques outperformed the base- line, underscoring their effectiveness in offline MOMO. The crossover operator demonstrated better performance than stochastic sampling due to its ability to combine existing high-quality molecules to create diverse offspring molecules. Most importantly, StitchNet achieved the best performance, demonstrating its effectiveness by leveraging the capability of a neural network integrate valuable chemical feedback.
4.2. Ablation Study
To investigate the impact of each key component in our framework-Rann-bassee Proxy (RP), StitchNet (SN), and Priority Sampling (PS) we conducted an ablation study.
Effects of rank- proxy. When RP was ablated and replaced with a score-based proxy, which is analogous to the vanilla proxy that directly approximates objective scores, we observed a noticeable drop in performance. This obser- vation highlights the efficacy of reformulating the proxy's 's task from score regression to pairwise classification, mak- ing the task easier and provide reliable feedback. Detailed investigations of score- and rank-based proxies are provided in Appendix D. In addition, we extended RP by employing multiple proxies, with the results presented in Appendix F.
Benefits of StitchNet. The ablation study highlighted the significant impact of SN in offline optimization process. By generating novel stitched molecules, SN provides valuable
7

---

Offline Model-based Optimization for Real-World Molecular Discovery
(a)
2D Pareto front
(b)
3D Pareto front
(c)
Score distribution
(d)
Diversity comparison
0.9
Ours
3500
StitchNet
0.8
w/o MolStitch
Number of unique substructure
3000
sampling
Stochastic
0.7-
0.6
0.6
0.6
2500
0.5
2000
GSK
60.4
0.4
0.4
1500
0.3
0.2
0.2
QED 0.2
1000
0.1
500
0.0
0.0
0.1
0.2
0.3
0.4
0.5 0.6 0.7
Stochastic sampling
StitchNet
JNK
JNK
Scaffold Carbon Skeleton
Figure 5. Visualizations of the 2D and 3D Pareto fronts (a-b), along with diversity analysis for StitchNet presented in (c-d).
Diversity analysis. In offline MOMO, promoting molecular diversity is crucial for identifying candidates with desirable properties while avoiding over-exploration of similar struc- tures. To assess the diversity of augmented molecules gen- erated by StitchNet in comparison to stochastic sampling, visualized their objective score distributions using vi- olin plots. As shown in Figure 5 (c), StitchNet exhibited broader and more varied score distribution, demonstrat- ing its capacity to provide a diverse range of augmented molecules for the generative model. We also evaluated the final molecules produced by the generative model fine-tuned with StitchNet against those from stochastic sampling, using diversity metrics that measure the number of unique sub- structures, specifically Bemis-Murcke (BM) scaffolds and carbon skeletons (Bemis & Mureko, 1996). As depicted in Figure 5 (d), the generative model fine-tuned with StitchNet exhibited greater diversity compared to stochastic sampling across both BM scaffolds and carbon skeletons. Additional diversity analysis for StitchNet is available in Appendix F.
Table 5. Performance comparison of various preference optimiza- tion techniques in offline multi-ohjecttvv molecular optimization.
Molecular objectives
Preference optimization
HV(f)
R2(J)
Baseline (REINVENT)
0.168 10.046
3.969110664
StitchNet & RLHF
0.232++00071
3.715++0.611
StitchNet & DPO
0.327=+0.081
3.015:+0.493
SStttchNet & IPO
0.344::0.022
2.955+10.533
SSitcchNet IPO& & RP (Ours)
0.352:10.080
2.953:10.51
Preference optimization. In our MolStitch framework, we fine-tuned the generative model using a process analogous to the preference optimization techniques employed in large language models. To evaluate different preference optimiza- tion techniques in offline MOMO, we explored alternatives such as RLHF (Ouyang et al., 2022), where the proxy model serves as reward model generate reward scores that are directly optimized. Other approaches involved removing the proxy by allowing the generative model to act as judge mnodel-as-a-jugeee to directly classify winning and losing molecules and update itself using DPO or IPO loss. As illus- trated in Table 5, our MolStitch consistently outperformed other techniques by constructing the separate bbsed proxy (RP) for molecule evaluation and updating the gener- ative model separately based this proxy feedback. This separation has shown be effective, as supported by recent studies (Singhal et al., 2024; Liu et al., 2024b), where main- taining a separate reward-rankin model helps to mitigate distributional shifts and enhance performance.
Scalarization analysis. In our MolStitch framework, we ini- tially employed linear scalarization due to its simplicity and foundational role in multi-objective optimization. However, we further investigated advanced scalarization techniques such as Chebyshev scalarization. Unlike linear scalarization, which combines objectives into a weighted sum, Chebyshev scalarization aims minimize the maximum deviation from ideal objective values. This characteristic enables it to better navigate non-convex Pareto fronts (Deb et al., 2016). To assess its compatibility with MolStitch, we conducted ex- periments integrating Chebyshev scalarization. As shown in tables provided in Appendix R, MolStitch with Chebyshev achieved comparable performance to the linear approach in the two-objective setting. However, it outperformed the linear approach in more complex three- and four-objective settings. We attribute this improvement to the growing com- plexity and non-convexity of the Pareto front as the number of objectives increases, where Chebyshev's emphasis on balancing extreme trade-off' provides greater robustness.
Pareto front visualization. To evaluate the impact of Mol- Stitch on solution quality, we visualized the Pareto front in both 2D and 3D objective spaces. As depicted in Figure (a- b). the Pareto front obtained from MolStitch dominated the baseline without MolStitch, indicating superior performance across all objectives. Notably, the solutions generated by MolStitch were concentrated in the upper right region of the Pareto front, signifying the effectiveness of molecular stitching process derived from Sti in offline MOMO.
8

---

Offline Model-based Optimization for Real-World Molecular Discovery
5. Related work
5.3. Offline Model-based Optimization (MBO)
5.1. Generative Models for Molecular Discovery
The most straightforward implementation of offline MBO is to use vanilla proxy to estimate objective scores and ap- plies gradient ascent to update the generative model directly. However, this approach often suffers from inaccurate proxy predictions, which can misguide the optimization process, To address this issue, various methods have been proposed.
In the past few years, a wide range of generative models has been proposed for molecular discovery, showing significant success in exploring chemical space and optimizing molec- ular properties. These models are commonly grouped into four main categories: genetic algorithms, sampling-base methods, reinforcement learning, and probabilistic methods.
Improving the proxy model. One line of research focuses on improving the accuracy and robustness of the proxy itself. For instance, COMs (Trabucco et al., 2021) employ adver- sarial learning to encourage the proxy to produce conserva- tive estimates. Similarly, IOM (Qi et al., 2022) introduces invariant representation learning through domain adaptation techniques to mitigate the impact of distributional shifts, Additionally, methods such as ICT (Yuan et al., 2023) and Tri- Tri-Mentoring (Chen et al., 2023a) leverage multiple proxy models to take advantage of ensemble learning.
Genetic algorithms operate by evolving a population of candidate molecules through mutation and crossover opera- tions, guided by a fitness function. Then, perfoorrforinn candidates are selected and propagated to subsequent gener- ations, with the aim of progressively improving population quality. A prominent example is GraphGA (Jensen, 2019), which demonstrates strong performance in navigating chem- ical space to generate molecules with desirable properties.
Sampling- based methods generate molecules by sampling from distributions that are biased toward regions of chemi- cal space likely to yield desirable molecular properties. A representative example is MARS (Xie et al., 2021a), which employs Markov Chain Monte Carlo (MCMC) sampling to explore chemical space and identify high-quality molecules.
Improving optimization algorithms. Another line of re- search focuses on enhancing the optimization strategies. For instance, BIB (Chen et al., 2023b) incorporates bidirectional learning by combining forward and backward mappings to generate input samples that are likely to yield high-quality outputs. BootGen (Kim et al., 2023) enhances optimization through bootstrapping strategy that iteratively augments the offline dataset with top-performing synthetic samples. RaM (Tan et al., 2025) employs learning-to-rank techniques to guide optimization based on relative rankings, which is conceptually similar to our rank-based proxy. However, the novelty of our framework is characterized by its combina- tion of molecular stitching, rank-based proxy, priority sam- pling, and preference optimization tackle offline MOMO,
Reinforcement learning formulate molecular generation as a sequential decision-making process, where an agent in- teracts with a chemical environment to construct molecular structures through a series of actions. A prominent example is REINVENT (Olivecrona et al., 2017), which employs agent to generate molecules in SMILES format in an autore- gressive manner. The agent learns to improve its policy over time by receiving reward feedhack based on the desirability of generated molecules with respect to target properties.
Extended related work section is provided in Appendix M.
Probabilistic methods also frame molecular generation as sequential process, similar reinforcement learning. How- ever, they differ in that actions are sampled from a learned probability distribution rather than a deterministic policy. A representative example is GetswNets (Jain et al., 2022), which aim to generate diverse and high-reward molecules by sampling from a distribution proportional to their reward.
6. Conclusion
In this study, we address two key challenges in real-world molecular discovery: offline molecular optimization and multi-objectiive molecular optimization (MOMO). To tackle these challenges, we propose the MolStitch framework. For offline molecular optimization, MolStitch uses StitchNet to generate novel stitched molecules by combining the de- sirable properties from two parent molecules in an offline dataset. These stitched molecules are then evaluated using rank-based proxy, which determines the more favorable molecule in a given pair. Using feedback from this proxy, the generative model is fine-tuned through preference opti- mization to enhance its capability to produce high-quality molecules. For MOMO, we further introduce priority sam- pling to explore trade-offs among multiple objectives more effectively. Through extensive experiments, we validate the efficacy of MolStitch addressing offline MOMO problem. Future work and limitations can be found in Appendix T.
5.2. Multi-Objective Molecular Optimization (MOMO)
The MOMO problem involves simultancously optimizing multiple, often conflicting, molecular objectives. Since a single solution cannot usually satisfy all objectives, the goal is to identify diverse set of Pareto-optimal molecules that represent trade-offs among objectives. A common approach is to use scalarization techniques such as weighted sums or Chebyshev-to to effectively aggregate multiple objectives into a single objective function. For example, GeneticGFN (Kim et al., 2024a) employs linear scalarization to manage multiple objectives and demonstrates strong performance.

---

Offline Model-based Optimization for Real-World Molecular Discovery
Acknowledgment
learning for biological sequence design. In International conference learning representations, 2019.
This work was supported by the Institute of Information & Communications Technology Planning Evaluation (ITP)), funded by the Korea government (MSIT), through the Artifi- cial Intelligence Graduate School Program Korea Univer- sity (No. RS-2019-111 1990099:;;;);; and by the National Research Foundation of Korea (NRF) grants funded by the Korea government (MSIT) (No. RS-2023-0021 (020200002222988 and (No. RS-2024-00415812)) It was also supported by the Ministry of Science and ICT, Korea government (MSIT), through the Information Technology Research Center (ITRC) support program
Anstine, D. M. and Isayev, O. Generative models as an emerging paradigm in the chemical sciences. Journal of the American Chemical Society, 145(16):8736-87550 2023.
Atiqur Rahman, M., Salajegheh, A., Anthony Smith, R., Lam, K.-y., et al. Braf inhibitor therapy for melanoma, thyroid and colorectal cancers: development of resistance and future prospects. Current cancer drug targets, 14(2): 128-143,40 2014.
Azar, M. G., Guo, Z. D., Piot, B., Munos, R., Rowland, M., Valko, M., and Calandriello, D. A general theoretical paradigm to understand learning from human preferences. International Conference on Artificial Intelligence and Statistics, pp. 4447-4455. PMLR, 2024.
Impact Statement
In this study, we address the offline multi-objective molecu- lar optimization problem, which has potential applications in drug discovery. We emphasize the responsible application of our methodologies, with strong focus on safety consid- erations. Although our framework enhances the efficiency of molecular optimization, it is crucial that all identified molecules must undergo experimental validation, safety assessments, and regulatory approval before being consid- ered for real-world deployment. We caution against rely- ing solely on computationally generated molecules without proper testing, as it could lead to unintended health or envi- ronmental risks. Furthermore, all datasets used in this study are publicly accessible and comply with ethical standards.
Bagal, V., Aggarwal, R., Vinod, P., and Priyakumar, U. D.
MolGPT: molecular generation using transformer- decoder model. Journal of chemical information and modeling, 62(9):2064-200 2021.
Bai, Y., Jones, A., Ndousse. K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training a helpful and harmless assistant with rein- forcement learning from human feedback. arXiv preprint (tlv:2204.05862. 2022.
Barbosa, F. and Horvath, D. Molecular similarity and prop- erty similarity. Current topics in medicinal chemistry, 4 (6):589-66)) 600, 2004.
Reproducibility Statement
We provide comprehensive information on the experimental settings, workflow, hyperparameters, and implementation details in Appendix L. Additionally, the source code for our proposed framework is available online at https:/ gi thub. com /Mol lecu 1 Teaaem MMol Motttt tch.
Bemis, G. W. and Murcko, M. A. The properties of known drugs. 1. molecular frameworks. Journal of medicinal chemistry, (9155))8877-289, 1996.
Bender, A. and Glen, R. C. Molecular similarity: a key tech- nique in molecular informatics. Organic & biomolecular chemistry, 2(22):3204 3218, 2004.
References
Beurel, E., Grieco, S. F., and Jope, R. S. Glycogen syn- thase 3 (gsk3): regulation, actions, and diseases. Pharmacology & therapeutics, 148: 14, 2015.
Achiam, J., Adler, S., Agarwal, S., Ahmad, I.., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08874, 2023.
Bickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S., and Hopkins, A. L. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98. 2012.
Alhossary, A.,
Handoko, S. D., Mu, Y., and Kwoh, C.- K. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31133)2214--2216, 2015.
Bilodeau, C., Jin, W., Jaakkola, T., Barzilay, R., and Jensen, K. F. Generative models for molecular discovery: Recent ad advances and challenges. Wiley Interdisciplinary Reviews: Computational Molecular Science, 12(5);e 1608, 2022.
Alvesalo, J., Siiskonen, A., Vainio, M. J., Tammela, P. S., and Vuorela, P. M. Similarity based virtual screening: a tool for targeted library design. Journal of medicinul chemistry, 49(7):2353-2333 2006.
Bogoyevitch, M. A. and Kobe, B. Uses for jnk: the many and varied substrates of the c-jun n-terminal kinases. Mi- crobiology and Molecular Biology Reviews, 70(4):1061- 1095, 2006.
Angermueller, C., Dohan, D., Belanger, D.,, Deshpande, R., Murphy, K., and Colwell, L.. Model-based reinforcement
10

---

Offline Model-based Optimization for Real-World Molecular Discovery
Bradley, R. A. and Terry, M. E. Rank analysis of incom- plete block designs: I. the method of paired comparisons. Biometrika, 39(344):324-355, 1952.
Chemical Information and Modeling, 63(11):323833247, 2023.
Croitoru, F-A.,, Hondru, V., lonescu, R. T., and Shah, M. Diffusion models in vision: A survey. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 45 19::000001,,6,, 2023.
Brockhoff, D., Wagner, T., and Trautmann, H. On the proper- ties of the r2 indicator. Proceedings of the 14th annual conference on Genefic and evolutionary computation, pp- 465-4722 2012.
Dao, M. C., Le Nguyen, P., Truong, T. N., and Hoang, T. N. Boosting offline optimizers with surrogate sensitiv- ity. In International Conference on Machine Learning. pp. 10072-10090. PMLR, 2024.
Calandriello, D., Guo, Z. D., Munos, R., Rowland, M., Tang, Y., Pires, B. A., Richemond, H., Le Lan, C., Valko, M., Liu, T., et al. Human alignment of large language models through online preference optimisation. In International conference on machine learning. PMLR, 2024.
Deb, K., Sindhya, K., and Hakanen, J. Multi-objective optimization. In Decision sciences, pp. 161-200. CRC Press, 2016.
Chakraborty, S., Qiu, J., Yuan, H., Koppel, A., Manocha, D., Huang, F., Bedi, A., and Wang, M. Maxmin-rlhf: Align- ment with diverse human preferences. In International Conference on Machine Learning, 2024.
Deshmukh, S., Elizalde, B., Singh, R., and Wang, H. Pengi: an audio language model for audio tasks. Advances in Neural Information Processing Systems, pp. 18090- 18108, 2023.
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, I.., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., al. survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1-45, 2024.
Eigenbrot, C. Structure, function, and activation of coagula- tion factor vii. Current protein and peptide science, 3(3): 287-299, 2002.
Chemingui, Y., Deshwal, A., Hoang, T. N., and Doppa, R. Offline model-based optimization via policy-guided gradient search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 11230-11239 2024.
Ertl, P. and Schuffenhauer, A. Estimation of synthetic acces- sibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of chem- informatics, 1:1-11, 2009.
Chen, C., Zhang. Y., Fu, J., Liu, X., and Coates, M. Bidi- rectional learning for offline infinite-width model-based optimization. In Advances in Neural Information Pro- cessing Systems, pp. 29454-29467, 2022.
Fink, K. B. and Göthert. M. 5-ht receptor regulation of neurotransmitter release. Pharmacological reviews, 59 (4):360-417, 2007.
Fromer, J. C. and Coley, C. W. Computer-aided multi- objective optimization in small molccule discovery. Pat- terns, 4(2), 2023.
Chen, C., Beckham, C., Liu, Z., Liu, X., and Pal, Parallel- mentoring for offline model-based optimization. In Ad- vances in Neural Information Processing Systems, pp. 76619-76636. 2023a.
Fu, J. and Levine. S. Offline model-based optimization via normalized maximum likelihood estimation. In Interna- tional conference on learning representations, 2021.
Chen, C., Zhang, Y., Liu, X., and Coates, M. Bidirectional learning for offline model- based biological sequence de- sign. In International Conference Machine Learning. pp. 5351-5366. PMLR, 2023b.
Fu, T., Xiao, C., Li, X., Glass, L. M., and Sun, J. Mimosa: Multi-constraint molecule sampling for molecule opti- mization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 125-133, 2021.
Cho, J. Wang, Y., Chen, R., Chan, K. S., and Swami, A. A survey on modeling and optimizing -bjei-obeecctvve systems. IEEE Communications Surveys & Tutorials, 19 (3):1867-1901, 2017.
Fu, T. Gao, W., Xiao, C., Yasonik, J.. Coley, C. W., and Sun, J. Differentiable scaffolding tree for molecule opti- mization. International Conference on Learning Repre- sentation, 2022.
Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv preprint arXiv: 141..355. 2014.
Gao, W., Fu, T., Sun, J., and Coley, C. W. Sample effi- ciency matters: a benchmark for practical molecular opti- mization. Advances in Neural Information Processing Systems, pp. 21342-21357. 2022.
Cieplinski, T., Danel, T., Podlewska, S., and Jastrzebski, S. Generative models should at least be able to design molecules that dock well: new benchmark. Journal of

---

Offline Model-based Optimization for Real-World Molecular Discovery
Giniatullin, R. S-hydroxytrrptammine in migraine: The puz- zling role of ionotropie 5-ht3 receptor in the context of established therapeutic effect of metabotropic 5-htl sub- types. British Journal of Pharmacology, 179(3):400-4155 2022.
Hu, X., He, T., and Wipf, D. New desiderata for direct preference optimization. In ICML. 2024 Workshop on Models of Human Feedback for Al Alignment, 2024b.
Jain, M., Bengio, E., Hermandez-Garciaa A., Rector- -Booor, J., Dossou, B. F., Ekbote, C. A., Fu, J., Zhang, T., Kil- gour, M., Zhang, D., et al. Biological sequence design with gflownets. In International Conference on Machine Learning, pp.
R., Verdín, S., Lavalle-Carrasee J., Molina-Frecheero N., Fiponoi,,, M., Carreón- Burciaga, R. G., and Bologna-Mo Moolina, R. Current con- cepts in ameloblastoma-targee oncogene serine/threonine kinase v600e mutation: Sys- tematic review. World journal of clinical oncology, 11(1): 31, 2020.
PMLR, 2022.
therapies in b-raf proto-
Jensen, J. H. A graph-baser based genetic algorithm and gener- ative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):357-3377. 2019.
Gu, A. and Dao. T. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:23/2.00752. 2023.
Jiménez-Luma, J., Grisoni, F., Weskamp, N., and Schneider, G. Artificial intelligence in drug discovery: recent ad- vances and future perspectives. Expert opinion on drug discovery, 16(9):949-959, 2021.
Gu, S., Xu, M., Powers, A. S., Nie, W., Geffner, T., Kreis, K., Leskovec, J., Vahdat, A., and Ermon, S. Aligning target-aware molecule diffusion models with exact en- ergy optimization. In Advances in Neural Information Processing Systems, 2024.
Jope, R. S., Yuskaitis, C. J., and Beurel, E. Glycogen synthase kinase-3 (gsk3); inflammation, diseases, and therapeutics. Neurochemical research, 32:577-595, 2007.
Guan, L., Yang, H., Cai, Y., Sun, L., Di, P., Li, W., Liu, G., and Tang, Y. Admet-score a comprehensive scoring function for evaluation of chemical drug-likeness, Med- chemcomm, 10(1):148-157) 2019.
Kamphausen, S., Höltge, N., Wirsching, F. Morys- Wortmann, C., Riester, D., Goetz, R., Thürk, M., and Schwienhorst, A. Genetic algorithm for the design of molecules with desired properties. Journal of Computer- Aided Molecular Design, 16:551-566 111115777400 2002.
Gunantara, N. A review of multi-objective optimization: Methods and its applications. Cogent Engineering, 5(1): 1502242, 2018.
Kim, H., Kim, M., Choi, S., and Park, J. Genetic-guided gfoowetss for sample efficient molecular optimization. In Advances in Neural Information Processing Systems, 2024a.
Guo, J. and Schwaller, P. Augmented memory: Sample- efficient generative molecular design with reinforcement learning. JACS Au, 2024a.
Kim, M., Berto, F., Ahn, S., and Park, Bootstrapped train- ing of score-conditioned generator for offline design of biological sequences. In Advances in Neural Information Processing Systems, pp. 77643-67666 2023.
Guo, J. and Schwaller, P. Satum: Sample-efficient genera- tive molecular design using memory manipulation. arXiv preprint arXiv:2405. 17066, 2024b.
Hall, C. A., Rapaport, S. I., Ames, S. B., DeGroot, J. A., Allen, F. S., and Ralston, M. A. A clinical and family study of hereditary proconvertin (factor vii) deficiency. The American Journal of Medicine, 37(2):172-181, 1964.
Kim, S., Choi, Y., Matsunaga, D. E.,, and Kim, K.-E. Stitch- ing sub-trajectories with conditional diffusion model for conl-conditioned offline rl. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 13160-13167, 2024b.
He, J., Nittinger, E., Tyrchan, C., Czechtizky, W., Patronov, A., Bjerrum, E. J., and Engkvist, O. Transformer-bat molecular optimization beyond matched molecular pairs. Journal of cheminformatics, 14(1):18, 2022.
Launay, J.-M., Herve, P., Peoc'h, h, K., Tournois, C., Calle- bert, J., Nebigil, C. G., Etienne, N., Drouet, I.., Humbert, M., Simonneau, G., et al. Function of the serotonin 5- hyyrroxytyptamiine 2b receptor in pulmonary hyperten- sion. Nature medicine, 8(10): 100::11991.... 2002.
Hoang, M., Fadhel, A., Deshwal, A., Doppa, J., and Hoang, T. N. Learning surrogates for offline blx optimiza- tion via gradient matching. In International Conference on Machine Learning, pp. 18374 18393, PMLR, 2024.
Lee, D.-J., Shin, D.-H., Son, Y.-.., and Kam, T.-F. Hybrid local- gnn for protein binding interface prediction via ensemble learning. In 2025 International Confer- ence on Electronics, Information, and Communication (ICEIC), pp. 1-4. IERE, 2025.
Hu, J., Wu, X., Wang, W., Zhang, D., Cao, Y., et al. Open- rlhf: An casy-to-use, scalable and high begehperffrmacce rlhf framework, arXiv preprint arXiv:2405. 143, 2024a.
12

---

Offline Model-based Optimization for Real-World Molecular Discovery
Lee, S., Jo, J., and Hwang, S. J. Exploring chemical space with score-based dittr--distribution generation. In Inter- national Conference on Machine Learning, pp. 18872- 18892. PMLR, 2023.
in Neural Information Processing Systems, 36:45856- 45869, 2023.
Olivecrona, M., Blaschke, T., Engkvist, O., and Chen, H. Molecular de-novo design through deep reinforcement learning. Journal of cheminformaties, 9:1-14, 2017.
Lee, S., Kreis, K., Veccham, S., Liu, M., Reidenbach, D., Paliwal, S., Vahdat, A., and Nie, W. Molecule generation with fragment retrieval augmentation. Advances in Neu- ral Information Processing Systems, 37:132463-132490, 2024.
Ouyang, L.,, Wu, J., Jiang, X., Almeida, D., Wainwright, L., Mishkin, P.. Zhang, C., Agarwal, S., Slama, K., Ray, A., al. Training language models follow in- structions with human feedback. In Advances in Neural Information Processing Systems, pp. 27730-27744, 2022.
Li, G., Shan, Y., Zhu, Z.,, Long, T., and Zhang, W. Diffstitch: Boosting offline reinforcement learning with diffusion- based trajectory stitching. In International Conference on Machine Learning, 2024.
Pal, A., Karkhanis, D., Dooley, S., Roberts, M., Naidu, S., and White, C. Smaug: Fixing failure modes of pref- erence optimisation with dpo-positive. arXiv preprint wrXiv:2402.13225 2024.
Liu, S., Luo, W., and Wang, Y. Emerging role parp-1 I and parthanatos in ischemic stroke. Journal of neurochem- istry, 160(1):74-9)) 2022.
Payton, A., Roell, K. R., Rebuli. M. E., Valdar, W., Jaspers, I., and Rager, J. E. Navigating the bridge between wet and dry lab toxicology research to address current challenges with high-dimensional data. Frontiers in Toxicology, 5: 1171175, 2023.
Liu, S., Dai, H., Zhao, Y., and Liu, P. Preference opti- mization for molecule synthesis with conditional residual energy-based models. In International conference on machine learning. PMLR, 2024a.
Qi, H., Su, Y., Kumar, A., and Levine, Data-driven offline decision-making via invariant representation learning. In Advances in Neural Information Processing Systems, pp. 2022.
Liu, T., Zhao, Y., Joshi, R., Khalman, M., Saleh, M., Liu, J., and Liu, J. Statistical rejection sampling improves preference optimization. In International conference on learning representations, 2024b.
Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, D., and Finn, C. Direct preference optimization: your language model is secretly a reward model. In Advances in Neural Information Processing Systems, pp. 53728- 53741, 2023.
Loiodice, S., Nogueira da Costa, A., and Atienzar, F. Cur- rent trends in in silico, in vitro toxicology, and safety biomarkers in early drug development. Drug and chemi- cal toxicology, 42(2):
2019.
M Ghari, P., Tseng, A., Eraslan, G., Lopez, R., Biancalani, T., Scalia, G., and Hajiramezanali, E. Gflownet assisted biological sequence editing. Advances in Neural Infor- mation Processing Systems, 37:106841-106669, 2024.
Ray Chaudhuri, A. and Nussenzweig, A. The multifaceted roles of parpl in dna repair and chromatin remodelling, Nature reviews Molecular cell biology, 18(10):110621, 2017.
Maggiora, G., Vogt, M., Stumpfe, D., and Bajorath, Molecular similarity in medicinal chemistry: miniper- spective. Journal of medicinal chemistry, 57(8):3186 3204, 2014.
Resnick, L. and Fennell, M. Targeting jnk3 for the treatment of neurodegenerative disorders. Drug discovery today, 9 (21):932 939, 2004.
Robinson, M. T., Rabinstein, A. A., Meschia, J. F., and Freeman, W. D. Safety of recombinant activated factor vii in patients with the central nervous system. Stroke, 41(7):1459-1463, 2010.
Malkin, N., Jain, M., Bengio, E., Sun, C., and Bengio, Y. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35:5955-599 2022.
hemorrhages of
Marler, R. T. and Arora, S. The weighted sum method for multi oujet--b optimization: new insights. Structural and multidisciplinary optimization, 41:853-8622 2010.
Rosasco, L.. De Vito, E., Caponnetto, A., Piana, M., and Verri, A. Are loss functions all the same? Neural compu- tation, 1665):1063-1076, 2004.
Minka, T. Estimating dirichlet distribution, 2000.
Rouleau, M., Patel, A., Hendzel, M. J., Kaufmann, S. H., and Poirier, G. G. Parp inhibition: Parpl and beyond. Nature reviews cancer, 10(4):293-301, 2010.
Nguyen, T., Agrawal, S., and Grover, A. Expt: Synthetic pretraining for few-shot experimental design, Advances
13

---

Offline Model-based Optimization for Real-World Molecular Discovery
Royer, A., Blankevoort, T.., and Bejnordi, B. E. Scalarization for multi-task and multi-domain learning at scale. In Advances in Neural Information Processing Systems, pp. 16917-16941, 2023.
Son, Y.-H., Shin, D.-H., Lee, D.-J., and Kam, T.-E. Molec- ular optimization with mamba-based gflownet. In 2025 International Conference Electronics, Information, and Communication (ICEIC), pp. 1-4. IEEE. 2025.
Sanz-Garcia, E.,, Argiles, G., Elez, E., and Tabernero, J. Braf mutant colorectal cancer: prognosis, treatment, and new perspectives. Annals of Oncology, 28(1 (8(11)22688-661) 2017.
Stanton, S., Maddox, W., Gruver, N., Maffettone, P., De- laney, E.,, Greenside, P., and Wilson, A. G. Accelerating bayesian optimization for biological sequence design with denoising autoencoders. In International Conference on Machine Learning, pp. 20459-20475 PMLR, 2022.
Seavey, M. M. and Dobrzanski, P. The many faces of janus kinase. Biochemical pharmacology, 830))113361145, 2012.
Sterling, T. and Irwin, J. J. Zinc 15-ligand discovery for everyone. Journal of chemical information and modeling,
Senkevitch, E. and Durum, S. The promise of janus kinase inhibitors in the treatment of hematological malignancies. Cytokine, 98:33 41, 2017.
Tan, R.-X., Xue, K., Lyu, S.-H., Shang, H., Wang, Y., Wang, Y., Fu, S., and Qian, C. Offline model-based optimiza- tion by learning to rank. International Conference Learning Representation, 2025.
Shin, D.-H., Ko, D.-H., Han, J.-W., and Kam, T.E. Evo- lutionary reinforcement learning for automated hyper- parameter optimization in EEG classification. In 2022 10th International Winter Conference on Brain-Computer Interface (BCI), pp. 1-5. IEEE, 2022.
Tang, Y., Guo, Z. D., Zheng, 7.,, Calandriello, D., Munos, R., Rowland, M., Richemond, P. H., Valko, M., Pires, B. Å., and Piot, B. Generalized preference optimization: A unified approach to offline alignment. In International conference machine learning. PMLR, 2024.
Shin, D.-H., Son, Y.HH., Kim, J.-M., Ahn, H.-J., Seo, J.- H., Ji, C.-H., Han, J.-W., Lee, B.-J., Won, D.-O., and Kam, T.-E. MARS; multiagent reinforcement learning for spatial spectral and temporal feature selection in EEG-based BCI. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 54(5):3084-3096 2024a.
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.099: 009888 2023.
Shin, D.-H., Son, Y.-H., Lee, D.-J., Han, J.. and Kam, T.-E. Dynamic many-objective molecular optimization: Unfolding complexity with objective decomposition and progressive optimization. In Proceedings of the Thirty- Third International Joint Conference Artificial Intelli- gence (IJCAI), pp. 6026-6034, 2024b.
Trabucco, B., Kumar, A., Geng, X., and Levine, S. Conser- vative objective models for effective offline model-based optimization. In International Conference Machine Learning, pp. 10358- PMLR, 2021.
Trabucco, B., Geng, X., Kumar, A., and Levine, S. Design- bench: Benchmarks for data-driven offline model-based optimization. In International Conference Machine Learning, pp.
Shin, D.-H., Lee, D.-J., Han, J..W., Son, Y.-H., and Kam, T.-E. Population-base based evolutionary search for joint hyperparameter and architecture optimization in brain- computer interface. Expert Systems with Applications, 264: 12583,, 2025a.
PMLR, 2022.
Tripp, A., Simm, G. N., and Hernández-Lobate J. M. A fresh look de molecular design benchmarks. In NewrIPS 2021 AI for Science Workshop, 2021.
Shin, D.-H., Son, Y.H.., Lee, D.-J., and Kam, T.-E. De- signing selective drugs: Multi-objective optmizztiion to mitigate off-target effects. In 2025 International Confer- ence on Electronics, Information, and Communication (ICEIC), pp. 1-5. IEEE, 2025b.
Van Laarhoven, P. J., Aarts, E. H., van Laarhoven, J., and Aarts, E. H. Simulated annealing. Springer, 1987.
Weininger, D. Smiles, chemical language and information
Singhal, P., Lambert, N., Niekum, S., Goyal, T., and Durrett, G. D2po: Discriminator-guinn guided dpo with response evalu- ation models. arXiv preprint arXiv:2405.00 01511, 2024.
system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36, 1988.
Son, Y.-H., Shin, D.-H., and Kam, T.-E. FTMMR: Fu- sion transformer for integrating multiple molecular rep- resentations. IEEE Journal of Biomedical and Health Informatics, 2024.
Xie, Y., Shi, C., Zhou, H., Yang, Y., Zhang, W., Yu, Y., and Li, L. MARS: Markov molecular sampling for multi- objective drug discovery. In International conference on learning representations, 2021a.
14

---

Offline Model-based Optimization for Real-World Molecular Discovery
Xie, Y., Shi, C., Zhou, H., Yang, Y., Zhang, W., Yu, Y., and Li, L. Mars: Markov molecular sampling for multi- objective drug discovery. In International Conference Learning Representations, 2021b.
Xiong, W., Dong, H., Ye, C., Wang, Z., Zhong, H., Ji, H., Jiang, N., and Zhang, T. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint, In International Conference on Machine Learning. PMLR, 2024.
Xue, K., Tan, R., Huang, X., and Qian, C. Offline multi- objective optimization. In International conference machine learning. PMLR, 2024.
Yamaoka, K., Saharinen, Pesu, M., Holt, V. Silven- noinen, O., and O' Shea, J. J. The janus kinases (jaks). Genome biology, 5:1-6, 2004.
Yu, S., Ahn, S., Song, L., and Shin, Roma: robust model adaptation for offline model-based optimization. Ad- vances in Neural Information Processing Systems, pp. 4619 4631, 2021.
Yuan, Y., Chen, C., Liu, Z., Neiswanger, W., and Liu, X. Importance-awe aware co-teaching for offline model-based optimization. In Advances in Neural Information Pro- cessing Systems, pp. 55718 55733,, 2023.
Yusuf, M. Insights into the in-silico research: current sce- nario, advantages, limits, and future perspectives. Life in Silico, 1(1):13-25, 2023.
Zhang, J., Zhang, J., Li, H., Chen, L., and Yao, D. Dual- target inhibitors of parp1 in cancer therapy: A drug dis- covery perspective. Drug Discovery Today, 2837):006077, 2023.
Zhang, R. and Golovin, D. Random hypervolume scalar- multi-objective objective black box optimiza- tion. International conference on machine learning, pp. 11096-1115. PMLR, 2020.
izations for provable multi
Zinkevich, M. Online convex programming and generalized infinitesimal gradient ascent. In International Conference on Machine Learning, pp. 928-936, 2003.
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., and
Da Fonseca, V. G. Performance assessment of multiobjec- tive optimizers: An analysis and review. JEEE Transac- tions on evolutionary computation, 7(2): 17-12,, 2003.
15

---

Offline Model-based Optimization for Real-World Molecular Discovery
Appendix
A. Online and Offline settings for Molecular Discovery
(a) Online setting
Dry lab experiments (generative model)
Potential drug candidates
Wet lab experiments
Evaluate drug candidates (true score assessment)
(oracle function)
Chemical feedback to generative model
(b) Offline setting
Offine model-based optimization
Training proxy model
medel update
Generative model
Drug candidates
Proxy model
Given offline dataset
Proxy model
Augmented dataset
Proxy score assessment
Figure An illustration of online and offline settings for molecular discovery.
In this section, we delve into the detailed pipeline for online and offline settings in molecular discovery, using the specific case of in silico drug discovery combined with real-world wet lab experiments as an illustrative example.
Online setting. Traditionally, many in silico drug discovery methods have been based on the assumptions of the online setting (Jiménez-Luna et al., 2021). As depicted in Figure (a), the online setting begins in the computational or 'dry' lab, where generative model produces potential drug candidates that are predicted to be potent. Researchers then select the top K drug candidates apply specific filters to choose which drug candidates to advance. These selected drugs are sent to the wet lab, where they undergo physical biological experiments to validate their efficacy. Note that these wet lab experiments serve as true oracle function, which provides accurate assessments of drug potency and properties based on real-world testing. Once the wet lab experiments are complete, the results-trit score assessments- are sent back to the dry lab as chemical feedback. This feedback is then used to update the generative model, enabling produce more desirable and potential drug candidates in the next iteration. This iterative process continues until successful drug candidates are identified predefined criteria, such reaching certain optimization score, met.
Why offline setting? The main advantage of the online setting is its ability to continuously refine the generative model using feedback from the true oracle function. However, this feedback relies on real-world wet lab experiments, which are typically coneemios and costly. Therefore, querying the true oracle function for every drug candidate is often impractical, and safety concerns can further limit its use (Loiodice al., 2019; Yusuf, 2023). Even we assume these challenges are mitigated and resources are available query the true oracle function as needed, there still remains the challenge of significant time mismatch between the dry lab and the wet lab. The dry lab can generate new drug candidates within hours, but the wet lab evaluation-ineel inncluding chemical synthesis, purification, and biological testing or even months (Payton et al., 2023). This significant lag means that while the wet lab is engaged in lengthy experiments, in silico generative models in dry lab remain idle, leading to inefficiencies and underutilization of computational resources. To address these limitations, the offline setting has gained considerable attention in recent years (Xue et al., 2024). In the offline setting, the generative model can be trained using existing offline datasets without relying on continuous feedback from wet lab experiments. This would allow generative models to be continually improved while awaiting wet lab results,
take weeks
16

---

Offline Model-based Optimization for Real-World Molecular Discovery
enabling the generation of higher-quality candidate molecules for subsequent experimental rounds.
Offline setting. One of the most prevalent and widely adopted approaches for handling the offline setting is offline model-based optimization (MBO). As illustrated in Figure 6 (b), the process begins by training proxy model on the given offline dataset. This proxy model serves as a surrogate for evaluating drug candidates, as access to the true oracle function is not available in the offline setting. Once the proxy model is trained, the offiin MBO process is initiated to enable the training of the generative model without relying on real-world wet lab feedback. Specifically, the generative model produces new drug candidates, which are evaluated by the proxy model instead of being sent to the wet lab. The proxy model provides estimated proxy scores for these candidates, and these pairs of drug candidates and their proxy scores are stored in a buffer to create an augmented dataset. This augmented dataset is then utilized to update the generative model via gradient ascent, leveraging the proxy model's predictions. This iterative cycle continues until predefined criteria are met.
MolStitch. After introducing the online and offline settings in molecular discovery, we now highlight how our proposed framework, MolStitch, differentiates itself from conventional offline MBO methods for tackling the offline multi-objective molecular optimization (MOMO) problem. First, instead of using a generative model to produce augmented molccules, MolStitch employs StitchNet generate novel stitched molecules. In molecular discovery, promoting molecular diversity is crucial because structurally diverse molecules capable of producing similar biological effects help mitigate the development of resistance and improve the overall efficacy of treatments. As discussed in the main manuscript and Appendix F, we validated that StitchNet exhibits better molecular diversity compared to conventional augmentation derived from generative models. This is because StitchNet naturally promotes diversity by exploring novel combinations derived from existing parent molecules. Second, conventional offline MBO methods utilize a score- score-based proxy to approximate the true oracle function by regressing objective scores of molecules. However, accurately approximating these scores is challenging due to scarce data and the inherent difficulty of molecular property prediction tasks-a a challenge that is even more pronounced in the MOMO problem. In MolStitch, we develop a rank-based proxy that learns the ranking relationship between pairs of molecules based on desired properties, classifying which molecule is more favorable. This transformation simplifies the task for the proxy, enabling it to provide more reliable feedback by focusing on comparative assessments rather than exact score predictions. Lastly, because we use a rank-based proxy, we cannot fine-ture tune the generative model, REINVENT, using conventional ways that require reward scores for updates (e.g., a score-based proxy could directly regress the objective score and use it as a pseudo-reward). To address this, we introduce preference optimization techniques to fine-tune and update the generative model. Specifically, instead of using pseudo-reward scores, the preference optimization technique enables the model to increase the -ig-liiel of generating the preferred (winning) molecule while decreasing the log- lieeliooodood of generating the less preferred (losing) molecule. As shown in Table 5 of our manuscript, experimental results demonstrate that this preference optimization is effective and significantly improves performance. In summary, MolStitch differentiates itself from conventional offline MBO methods by promoting molecular diversity through the use of StitchNet, simplifying proxy modeling via a rank-based approach, and incorporating preference optimization techniques to fine-tune the generative model without relying on approximating objective scores. These novel components collectively contribute to a more efficient and effective approach for tackling the offline MOMO problem.
Our Framework:
B. Experimental Workflow for Offline Multi-Objectivy Molecular Optimization (MOMO)
Overall workflow. In this subsection, we aim to conduct an in-depth exploration and comparison of key components in offline MOMO. Specifically, our goal is to outline the critical components that should be considered for solving the offline MOMO problem, discuss the available options for each component, and explain the rationale behind our choices. Figure 7 provides a visual representation of the overall workflow for addressing the offline MOMO problem. The primary objective of offline MOMO to enhance the generative model's capability to generate molecules that surpass the best-known molecules in the offline dataset. To achieve this, the predominant approach is offline MBO, which involves training a proxy model, performing data augmentation, generating synthetic data, and subsequently training the generative model with this synthetic data under the guidance of the proxy model. Consequently, data augmentation is pivotal aspect of the offline MOMO problem, and we begin our discussion with this component.
Data augmentation. As highlighted in the main manuscript, we propose StitchNet as a neural network model designed for data augmentation, and demonstrate its effectiveness. However, we acknowledge that StitchNet is not the only viable option. Alternative approaches include stochastic sampling, where new molecules are randomly drawn from the generative model's 's learned distribution. Additionally, rule-based crossover operators from genetic algorithms can be employed to generate new offspring molecules by combining features from parent molecules.
17

---

Offline Model-based Optimization for Real-World Molecular Discovery
Offline Dataset
Data Augmentation
Generative model selection
Proxy Training
!) Stochastic sampling
"Score-based proxy Rank-based proxy 3) No proxy
1) REINVENT 2) Mamba 3) GFlowNets
Crossover
operator 9) StitchNet
Fine-tuning Generative Model
augmented molecules
Proxy Eualuation
Score-based regression
reward (score)
1) RLHF DPO
Rank-based classification
win/lose
IPO
3) Generative model-
as-a-judge
Figure illustration of the overall workflow for the offline molecular optimization process.
Proxy training and evaluation. After augmenting the synthetic data, the next step involves training proxy model to evaluate this augmented dataset. The most straightforward approach the score-based proxy (vanilla proxy), which directly approximates the scores of the true objective function. However, we anticipate that as the problem complexity increases, the vanilla proxy may encounter challenges and yield unreliable predictions. To mitigate this, we propose rank-based proxy that learns the ranking relationships between pairs of molecules based desired properties, thereby classifying which molecule is more favorable. This transformation from regression task to classification task simplifies the proxy's role, enhancing its reliability in providing feedback to the generative model. It is worth noting that proxy model is not always necessary; in some cases, the generative model itself can evaluate new synthetic data, mechanism referred to as the aaodel--s----uudge
Generative model selection. Several generative models are available for molecular optimization. In this work, we employ REINVENT as our main generative model due its widespread use and recognition various molecular optimization tasks. Nonetheless, recent advancements have introduced new generative models such as Mamba and GFlowNets. To ensure the robustness and versatility of our MolStitch, we also evaluate various backbone generative models within this framework.
Fine-tuning the generative model. With synthetic data, trained proxy model, and trained generative model in place, the final step involves fine-tuning the generative model using the synthetic data guided by the proxy model. This fine-tuning process can be considered analogous to the preference optimization process used in large language models. Therefore, we explore various preference optimization techniques within the context of offline MOMO. The first option is RLHE, where
18

---

Offline Model-based Optimization for Real-World Molecular Discovery
the proxy model serves reward model to generate rewards that directly optimized. Another option DPO, which bypasses reward modeling and focuses optimizing preferences directly. Lastly, IPO be applied extension of DPO, providing more theoretically sound and principled approach to preference optimization.
Overview of MolStitch components. Table 6 presents detailed summary of the components constituting the MolStitch framework, including its variants and the methods examined in our ablation studies. We hope that this table helps to understand the function of each component in our framework and facilitates a clearer understanding of structure of MolStitch and its variants.
Table Summary of our MolStitch framework components, its variants, and methods utilized in our ablation studies.
Data
Proxy Training
Generative Model
Proxy Evaluation
Experiment
Fine-tuning
Augmentation
MolStitch (Table
Rank-based
Rank-based
StitchNet
REINVENT
IPO
proxy
classification
Score-based proxy (Table 3)
Stochastic
Score-based proxy
Score-based regression
REINVENT
RLII
sampling
Stochastic sampling (Table 4)
Stochastic
Rank-based proxy
Rank-based classification
REINVENT
IPO
sampling
Crossover operator (Table 4)
Crossover
Rank-based proxy
Rank-based classification
REINVENT
IPO
operator
StitchNet RLHI
Rank-based proxy
Score-based regression
StitchNet
REINVENT
RLIIF
(Table 5)
StitchNet & DPO (Table
Generative model-as-a-judes
StitchNet
No proxy
REINVENT
DPO
StitchNet & IPO (Table
Generative model-as-a-judge
StitchNet
No proxy
REINVENT
Mamba + MolStitch
Lank-based proxy
Rank-based classification
StitchNet
Mamba
(Table 12)
GFlowNets MolStitch (Table 12)
Rank-hased proxy
Rank-based classification
StitchNet
GFlowNets
IPO
19

---

Offline Model-based Optimization for Real-World Molecular Discovery
C. Additional Results
C.1. Full HV Performance Results for the Docking Score Optimization Task
Results for HV performance. In our main results, we only presented performance evaluations using the Hypervolume (HV) indicator for select group of strong competing methods. Specifically, we highlighted Saturn and GeneticGFN, both of which demonstrated strong performance and represent recent advancements employing genetic algorithms, as well as BootGen and RaM, recognized as state-of-the-art offlin MBO methods for biological sequence design. Due to space constraints, our primary results focused on these prominent methods. However, in this subsection, provide the full results for the docking score optimization task, offering a comprehensive comparison across various competing methods.
Table Experimental results docking score optimization tasks under the full-offlnee setting using the HV indicator.
Target protein
parpl
jak2
braf
fa7
5htlb
Method
HV(()
HV(t)
HV(1)
HV(1)
HV(1)
REINVENT
0.515
0.407
0.500+10:008
0.414:10.006
0.509+00001
AugMem
0.532110.039
0.409
1..008
0.430+10.038
0.5014
Saturn
0.52810.00
0..00
1.0030
0.003
0.431::0.034
0.537-10.033
GeNeticGFN
0.539
0.476:10.00
0.508 10.005
0.441 :10004
0.523:10.00
Grad
0.003
0.48 1 1:0011
0.5 0.000
0.445
0.525
COMs
0.510=00010
0.478:=00014
0.505=00022
0.411=00.000
0. 0.509 008
IOM
0.520=0.009
0. 0.474::0.008
0.500=000013
0.41 0
0,519 0.002
RoMA
0.512=00010
0. 0.400=0.009
0.5 02000032
0.429
0.5 0.502330013
Ensemble Proxy
0.51710.008
0.479
0.501 1.0010
0.414 1.0006
0.507 00.008
BIB
0.514:100010
0.406
0.407
0.414
0.505
BootGen
0.544-10.032
0.406
0.504
0.436.10.000
0.545
ICT
0.516:20.005
0. 0.476:20.006
0.504
0.4 0.00
0,506=10.010
Tri-Mentoring
0.529
0. 00007
0.5 1 =00019
0.41 0
0.513
RaM
0.542++00228
0.528=0.034
0.424:10.008
0.525
MolStitch (Ours)
0.560=
0.515
0.554
0.4511:0061
0.575+0.081
C.2. R2 Performance Results for the Docking Score Optimization Task
Results for R2 performance. We present additional R2 performance results for the docking score optimization task in Table 8. Consistent with the findings in Table 7, our MolStitch framework demonstrated superior performance by achieving the lowest R2 indicator score compared to all competing methods.
Table 8. Experimental results on docking score optimization tasks under the full-offline setting using the R2 indicator.
Target protein
parp1
jak2
braf
fa7
Shtlb
Method
R2(L)
R2(4)
R2(()
R2(4)
R2(4)
REINVENT
1.466:0.090
1.88910.042
1.497:00004
1.791+10.03
1.454:10.054
AugMem
1.364
1.523
1.471
1.729
1.421
Saturn
1.376:00.53
1,501 100155
1.4
1.726:10.01
1.350:10.139
GeneticGFN
1.326=0.1448
1 1.589
1. 1.084=00002
1.701 20,228
1,410+40055
Grad
1.422+0.032
1.555
1.461 #0.036
1.750=#0.188 184
1,401
COMs
1. 1.448+00041
1. 1.568=00.899
1.467=0.109
16600000
1.459+0.045
IOM
1. 1..0
1 1.597.0.005
1.488 10.070
1.806
1. 10.16
RoMA
10.053
1 10.044
1..
1.
1 1.449 10.058
Ensemble Proxy
1..03
1 1.568
1.491 10.036
1..00....228
1 1..00.
BIB
1.425
1.503=000299
1.500=0.034
1.801
1.478
BootGen
1.320+0.136
1.507
1.420=0.030
1.712+00112
1.336+00184
ICT
1.228=0.024
1.591 1.591=0000 +00029
1.473+00100
18100=0.028
1. 1.472+00.045
Tri-Mentorigg
1.373=0.115
1..03
1 1.428=0.098
1.793+00003
1. 1.443+L00045
RaM
1.32340.112
1.5466100036
1.58
1.778:40.087
1. 1,401
MolStitch (Ours)
1.276=10.153
1.445
1.312-10.74
1.674:10.261
1.231
20

---

Offline Model-based Optimization for Real-World Molecular Discovery
C.3. Evaluating Molecular Optimization Methods Using Average Property Score of Top 10 and Top 100 Molecules
Results for APS performance. In our main results, we presented performance using the Hypervolume (HV) and R2 indicator metries, which are widely regarded as the most appropriate evaluation metrics for multi-objeccive optimization tasks. However, within the molecular discovery community, the average property score (APS) is another commonly used metric, specifically tailored for assessing molecular optimization methods. To provide more comprehensive assessment, we conducted additional experiments to report APS for various molecular optimization methods. The methods we evaluated include GraphGA (Jensen, 2019). which generates molecules by using rule-based crossover operations to combine features from parent molecules; MolGPT (Bagal et al.. 2021), which is suitable for offline settings as it does not require oracle calls during molecule generation; DST (Fu et al., 2022), which leverages a proxy model to facilitate precise functional group editing; and REINVENT (Olivecrona et al., 2017), our backbone generative model, known for its robust performance in molecular optimization tasks. Additionally, we also considered AugMem (Guo & Schwaller, 2024a), leading model in the PMO benchmark, Saturn (Guo & Schwaller, 2024b), which enhances sample efficiency in molecular design, and GeneticGFN (Kim et al., 2024a), which integrates GFlowNets with genetic algorithms to achieve state-of of-the-art performance across various molecular optimization tasks. In this experiment, we calculated the APS of the top 10 and top 100 molecules generated by each method and reported the mean APS, As shown in Table 9, our MolStitch framework consistently outperformed all competing methods, even when evaluated with the molecule sppecciiis metric. This result demonstrates the robustness and superiority of MolStitch across diverse evaluation criteria.
Table 9. Experimental results on molecular property optimization tasks under the full-offline setting using the average property score.
Molecular objectives
GSK33+JNK3
GSK3,J+NK3++EED
CSS339+JN3++EE-SSA
Method
top10 (f)
top100 (1)
top10 (1)
top100 (1)
top10 (1)
top100 (1)
REINVENT
0.515-40:076
0.312++0.36
0,464100088
0,564 0.383++0.005
0.491
AugMem
0.558:00006
0.515++0041 0.374
0.407
0.579=00015
0.505=+0.005
MolGPT
0.305:10.027
1.199
0.461
0.380.1..005
0.548110.014
0.485
GraphGA
0.406.10.07
0..05
0.512.10.048
0.405110.012
0.593
0.50110.010
DST
0.456:10.058
0.305
0,531 1100599
0.451.1..00
0.601 1.0.027
0.53910..02
Saturn
0.559+00074
0,358=000037
0.546==0.032
0.443=00041
0.513+00041 0.608
GeneticGFN
0.540=+0.077
0.379=#0078
0.548++0.088
0.451
0.599=00027
0.524
MolStitch (Ours)
0.627=40006
0.432++0.099
0.591+:0.000
0.468==00016
0.671=00041
0.564++0022
C.4. Semi-Offline Optimization
Definition of semi-offline optimization. Semi-offine optimization, also referred to batch hybrid learning (Xiong et al., 2024), is an optimization approach that bridges the gap between offline and online optimization. In this semi-offline setting, models are trained on a combination of pre-existing offline datasets and periodically collected new data, enabling periodic updates without the need for continuous or real-time oracle queries. Unlike the full-offline setting, where the model is trained exclusively on a static offline dataset, the semi-offline setting allows for the periodic incorporation of new data in large batches, facilitating a more dynamic learning process. This semi-offline optimization is particularly useful in scenarios where obtaining new data in real-time is either too costly or logistically challenging, yet some level of interaction or adaptation new data is beneficial.
Semi-offline optimization in LLMs. Semi-offline optimization has gained considerable attention in the field of large language models (LLMs). Several studies (Bai et al.. 2022: Touvron et al., 2023) have implemented a strategy of iteratively applying the RLHF process on a weekly cadence. This involves periodically deploying updated RLHF models to interact with users or crowdworkers to collect new preference data. The models are then fine-tuned with this feedback on a regular schedule. Recently, (Xiong et al., 2024) further extended this approach by formulating as a batch hybrid framework, establishing more general setting for the hybrid learning process.
Experimental setup for semi-offline optimization. Motivated by these practical applications, conducted additional experiments on MPO tasks under the semi-offline setting. We began by constructing an initial offline dataset using 5,000 oracle calls. In contrast to the full-offline setting, where all remaining 5,000 oracle calls were used for evaluation, the semi-offline setting employed a different allocation strategy. Specifically, we allocated 2,500 oracle calls for the periodic integration of new molecular data in large batches. This allocation enabled the generative model to iteratively update and adapt based on the newly acquired data, The remaining 2,500 oracle calls were reserved for the final evaluation.
21

---

Offline Model-based Optimization for Real-World Molecular Discovery
Table 10. Experimental results on molecular property optimization tasks semi-ofiline setting. with the best values in bold.
Molecular objectives
GSK3,9+INK3
+SK33+INK3+OED
GSK39++NK3+QED-SA
Method
HV(t)
R2(()
HV(t)
R2(()
HV(1)
R2(4)
REINVENT
0.581=
0.694+0.1109
0.208
2.372+40.300
0.175=+0.064
4.053-+0.747
AugMem
0.636=0.063
0,602=t01113
0.348=00075
18888++0.277
0.292:40.08
3.225=+0.650
GraphGA
0.501
0.819+0.113
0.392=+0.12
1.623++0.277
0.265++0.000
3,493++0.557
Saturn
0.623=0.049
0.621 1:0.06
0..28110.040
1.0.1100.60
0.382.10.088
2.686
GeneticGEN
0.642-10.065
0.592:10.107
0.41 1..11110.123
1.660.10.425
0.361 1:0.086
2.879910.569
Grad
0.584_0.075
0.708:10.136 136
0.21 0.000
2.458:10.371
0.1 0.30003
455
COMs
0.717:t0.10S 0.551
0.219
2.505=+:0351
0.186:
3.956=t0.505
IOM
0.603=0.061
0.647-10.081
0.221 +00077
2.349:+0.395
0.205 +00066
3. 3.899400.621
RoMA
0.588=0.067
0.680++0.109
0.0000
2.414:10.258
0.180
4.105
Ensemble Proxy
0.602=00.844
0.648
0.227
2.435-+0.332
0.216:10.009
3.730++0553
BIB
0.563=0.066
0.713=20122
0.215
2.440:100388
0..09200070
4.062:10.755
BootGen
0.608=0.057
0.646=t0.098
0.003
2.399++0.462
0.219
3.924
ICT
0.601 ::0.78
0.662.10.13
0.216110.089
2.455.10.389
0.185 1:0008
1.0454
Tri-Mentoring
0.592:10.078
0.67841..14
0.219
2.467.10.241
0.206
3. 3.966100.03
MolStitch (Ours)
0.689_10.00
0.514
0.539 1.0045
1.2157
0.493 0.005
2.014
Table 11. Performance of various preference optimization techniques for the semi-offline setting.
Molccular objectives
GSK3$+JNK3
GSK3/++NK3+QED
GSK33++NK3+QE++SA
Method
HV(t)
R2(L)
HV(f)
R2(4)
HV(1)
R2(4)
Baseline (REINVENT)
0.462=+0.13
0.921+00259
0.196:10.083
2.646++0327
0.168+00046
3.969#00664
StitchNet & RLHF
0.675.10.059
0.506.10.091
0..08
1.540-10.21
0.383
2.647:10463
StitchNet & DPO
0.685
0.520:10.083
0.507
1.342-40.221
0.447:10.000
2.320+10.31
StitchNet & IPO
0.681 1:0002
0.521
0.527:10.055
1.256-10.13
0.462:10.055
2.187:10:99
StitchNet & IPO & RP (Ours)
0.6894
0.514:10.073
0.539:10.04
1.238:+0.157
0.493:10.050
0.012
Results for semi-offline optimization. As illustrated in Table 10, our MolStitch framework consistently outperformed all competing methods under the semi-offline setting. Notably, we observed a general improvement in performance compared the full-offline setting, shown in Table I of the main manuscript. This finding highlights the benefits of incorporating periodic new data, as it enables the generative model to be fine-tuned and trained newly acquired samples, thereby further enhancing its optimization capabilities. Consistent with the trends observed in the full-offline setting, Saturn and GeneticGFN maintained strong performance among competing methods, highlighting the effectiveness of genetic algorithms in offline MOMO. Their success could be attributed to the inherent strengths of genetic algorithms in maintaining population diversity and effectively exploring the Pareto front through crossover operations. This finding aligns with our framework, which employs a mechanism analogous to crossover, but with the added advantage of incorporating chemical feedback. Additionally, we conducted experiments for preference optimization techniques under the semi-offline setting, as depicted in Table 11. The trends observed were similar those in the full-offline setting, our MolStitch consistently achieved the highest performance among all competing methods. While RLHF performed well on the two-objective scenario, its performance declined significantly as the number of objectives increased. Both DPO and IPO demonstrated strong performance, with IPO showing a slight edge over DPO.
22

---

Offline Model-based Optimization for Real-World Molecular Discovery
C.5. Evaluating Mamba and GelowNets as Additional Backhone Models
Various backbone models. In this work, we chose REINVENT as our backbone generative model due to its widespread use and reputation one of the nop-perfomning models for various molecular optimization tasks. However, as previously mentioned, Saturn and GeneticGFN demonstrated strong performance in numerous offline MOMO experiments. Since these methods utilized Mamba and GelowNets as their respective backbone models, we conducted additional experiments using Mamba and GelowNets as the backbone generative model for our MolStitch framework.
Results for backbone models. As illustrated in Table 12, we report the performance of each backbone generative model REINO--REEE,,,, Mamba, and GFlowNets- on MPO tasks under the full-offline setting. alongside the performance of integrating either our rank-based proxy or MolStitch framework with each backbone model (e.g., REINVENT + MolStitch). Similarly, Table 13 presents the performance of the backbone generative models and their respective integrations with MolStitch under the semi-offline setting. As shown in Table 12, both the rank-based proxy and the MolStitch framework provide performance improvements across various generative models. However, the integration with the rank based proxy still falls short compared to the full MolStitch framework, emphasizing the additional henefits brought by StitchNet and priority sampling. Notably, Mamba + MolStitch and GFlowNets + MolStitch outperformed REINVENT + MolStitch in both three-objective and four-objective scenarios. This superior performance could be attributed to the greater capacity of Mamba and GFlowNets to manage the increased complexity associated with optimizing multiple objectives beyond two. Overall, the consistent performance improvements across different backbone generative models under both full-offline and semi-offline settings demonstrate the robustness and versatility of our MolStitch. Moreover, these additional results highlight the MolStitch's 's ability to seamlessly integrate with range of backbone models, demonstrating its adaptability and robustness beyond a single model architecture.
Table 12. Performance comparison of different generative models on molecular property optimization tasks under the full-ofitine setting. GSK38++NK3+OED+SA
Molecular objectives
GSK3/+JNK3
GSK3,+INK3+QED
Method
HV(t)
R2(()
HV(1)
R2(4)
HV(t)
R2(L)
REINVENT
0.462.10.133
0.921
0.196:10.083
2.646-10.327
0.168
3.969.10.64
Rank-based Proxy
0.545
0.773-10. 120
0.319:0.059
1.928
0.251
3.504-10.64
+ MolStitch (Ours)
0.579:00070
0.698 128
0.403+00065
1.649-0.259
0.352:40.800
2.953:20.571
Mamba
0,531 1:0008
0.785:10.199
1..77 0.203
0.281 :10.58
3.339
+ Rank-based Proxy
0,758++0.10 0.508
105 0.761++0.12
0.327 100
1,946+00400
0.281=00077
3.817=0486
+ MolStitch (Ours)
0.544=0.071
0.407=00077
1.617=0.199
0.361
2.893=00424
GFlowNets
0.482-10.073
0.869 117
1.990-10.36 0.309
0.2377
3.630.L0.453
+ Rank-based Proxy
0.522=:0.040
0.805--0.085
0.364410.070
1.809
0.323=400054
2.953=20,004
+ MolStitch (Ours)
0.525
0.770:10.111
0.415
1.685
0.366:10.088
2.708
Table 13. Performance comparison of different generative models on molecular property optimization tasks under the semi-offline setting.
Molecular objectives
GSK3
GSK38+JNK3+QED
GSK38++NK3+QE++SA
Method
HV(t)
R2(1)
HV(1)
R2(L)
HV(t)
R2(L)
REINVENT
0.581 10.057
0.694-10.109
2.372 0.208
0.175.10.64
4.053.
+ MolStitch (Ours)
0.689.10.04
0.514-10.033
0.539
1.238
0.493
2.014-10.02
Mamba
0.623
1:0006
0.428
1.561
0.382
2.686110.510
+ MolStitch (Ours)
0.653:20.046
0.580.+0.090
0.485
1.430
0.434
2.385
GFlowNets
0.642-10.065
0,592-10.10
0.414:10.123
1.660:1..22
0.361:10.086
1100086
2.8599
+ MolStich (Ours)
0.658.10.068
0.563:10.108
0.5791.0.04
1.137
0.482
2.181
1.0.38
23

---

Offline Model-based Optimization for Real-World Molecular Discovery
C.6. Comprehensive Ablation Study for the Molecular Property Optimization (MPO) Task
Comprehensive Ablation Study Results. In the main manuseript, we presented ablation study results focusing only on the four-objective (GSK3)A-N scenario due to space constraints. Here, we provide complete ablation study, including two- and three-objective scenarios. for thorough evaluation. As shown in Table 14, the results clearly illustrate the effectiveness of our framework's 's key components: rank- lan--ased proxy (RP), StitchNet (SN), and priority sampling (PS), The comprehensive results consistently show improvements across all objective scenarios, underscoring the critical role of each component in driving superior performance for the offline MOMO problem.
Table 14. An ablation study for key components in our framework:
Proxy (RP), StitchNet (SN), and Priority Sampling (PS).
Ablation
GSK333JJNK3
GSK33+JNK3+OED
GSK3/+INK3+0E44SA
RP
SN
PS
HV(())
R2(()
IVV(1)
R2(()
IV((1)
R2(()
0.404
0.857 1.0.226
10.05
2.
0.171:10.00
4.176.10.319
v 0.003
0.78010.106
0.269.10.081
2..331
0.193 10.05
4.134
v
v
0.505=00049
0.824
0.277=00088
2.195
0,220=00054
3.835++04483
0.545=00063
0.773++01120
0.319+000059
1.928=00314
0,25 #00084
3.504+0.634
0.573=00078
0.638
0,337:+00008
1.967=#.311
0.289=00096
3.317=207113
0.579:10.070
0.698110.128
0.403:10.065
1.649
0.000
2.953
C.7. Comprehensive Results for Performance Comparison of Data Augmentation Techniques in Offline MOMO
Comprehensive Evaluation of Data Augmentation Techniques. In the main manuscript, focused exclusively on the four-objective scenario (GSK3 due to space limitations. Here, extend the analysis to include two- and threc-objective scenarios, offering a broader perspective on the performance of various data augmentation techniques. As summarized in Table 15, the findings clearly establish StitchNet as the most effective technique, consistently outperforming alternative approaches and demonstrating its ability to enhance performance across all objective scenarios in the offline MOMO problem.
Table 15. Performance comparison of different data augmentation techniques offline MOMO.
Molecular objectives
GSK38+JJK3
GSK38+INK3+QED
GSK3)-+NN3+QED+SA
Augmentation
HV(t)
R2(1)
HV(t)
R2(()
HV(t)
R2(J)
Baseline (REINVENT)
0.462210.133
0.921:10.55
0.196220088
2.646:10.227
0.168
3.96911.664
+ Stochastic sampling
0.545_100033
0.773+40.12 120
0.319=0.059
1.928
0.251 1..084
3.504=00633
Crossover operator
0.540=0.088
087
0.367=0.062
1.793:10.245
0.302:10.072
3.110
StitchNet (Ours)
0.079
0.698
0.403 0.065
1.649
0.352:100000
1.573
C.8. Comprehensive Results for Various Preference Optimization Techniques in Offline MOMO
Expanded Evaluation of Preference Optimization Techniques. The main manuscript presented results for the four- objective scenario (GSK)S+-NK3+OED+SAA) to maintain concise focus. In this section, we broaden the scope to include evaluations for two- and three-ohjective scenarios, offering more comprehensive assessment of preference optimization methods. As depicted in Table 16, our proposed approach (constructing the separate rank-based proxy (RP) and updating the model separately based on the proxy feedback) demonstrates a clear advantage, consistently outperforming other preference optimization techniques. These results underline its effectiveness and reliability across varying objective scenarios within the offline MOMO problem.
Table 16. Performance comparison of various preference optimization techniques in offline MOMO.
Molecular objectives
GSK38+JNK3
GSK38+INK3+QED
Method
HV(1)
R2(()
HV(T)
R2(J)
HV(1)
R2(J)
Baseline (REINVENT)
0.462 133
0.921 +00259
0.196 0..003
2.646-.0.227
0.168-40.046
3.969-10.664
StitchNet & RLHF
=0.05
0.742
0.303-10.087
2.012+4.318
0.232++0.07
3.715:10.111
StitchNet & DPO
0.557=0.094
0.747
0.363
1.843
0.327-10.81
3.015:10499
StitchNet & IPO
0.552
0.746:10.106
0.385-10.062
1.755
0.344-10.082
2.955-10.533
+ Stitchhet & IPO & RP (Ours)
0.579=0.070
0.698
0.403-:0.065
1.649=0.259
0.352++0.080
0.953-40.571
24

---

Offline Model-based Optimization for Real-World Molecular Discovery
D. Detailed Analysis of Rank-based Proxy
In this section, we provide an in-depth analysis both rank-based and score-based proxies. Our study suggests that formulation of rank-based proxy simplifies the proxy's task, thereby enabling it to deliver more reliable feedback the generative model. To further explore this, we delve deeper into the performance of each proxy type, examining whether the rank-based proxy truly surpasses the score-based proxy in handling complex multi-objective molecular optimization tasks.
Proxy models. In the context of utilizing proxy models, they offer distinct advantages, but they also present notable challenges. Specifically, Grad is built upon REINVENT and incorporates vanilla score-based proxy that directly approximates objective scores. As shown in Table of our main manuscript, while Grad outperforms the baseline REINVENT, its performance gains gradually diminish as number of objectives increases from two four. This suggests that with the rise the number of objectives, the problem complexity increases, causing the vanilla proxy to struggle to accurately approximate the objective scores, In contrast, our framework demonstrates particularly strong performance the three and four objective scenarios, which highlights the effectiveness of reformulating the proxy model's 's task from direct property score regression to pairwise classification.
Stitch Proxy Score
Stitch Praxy Score
Stitch Praxy Score
20
30
60
2 15-
Densty
Density
20-
40
10
5
8.0
8.0
8.0
0.5
Score
Score
Score
(a) GSK3,4JNK33
(b) GSSE3/+INK3+QED
(c) CSK39HHK3++HD4SSA
(2 objectives)
(3 objectives)
(4 objectives)
Figure 8. Distribution comparison of true objective scores (red) and score-based proxy model predictions (blue) for stitched molecules across varying numbers of objectives. As the number of objectives increases, the score-based proxy model's predictions show less variability and exhibit sharper central peak, failing to accurately represent the true score distribution.
Score-based proxy. As shown in Figure 8, we visualize the distribution of the true scores for the stitched molecules alongside the predicted scores from the score-based proxy model. Compared to the distribution of true objective scores, the predictions made by the score-based proxy model are significantly more confined to narrow range. This issue becomes more pronounced as the number of objectives increases, with the score-based proxy model's predictions showing even less variability and stronger central peak, failing to represent the true score distribution accurately. Therefore, this result indicates that the score-based proxy model fails to provide meaningful feedback to the generative model, potentially leading to suboptimal optimization. To address these limitations, we propose rank-based proxy model that learns the relative ranking between pairs of molecules based on desired properties, determining which molecule is more favorable. This approach bypasses the direct approximation true objective scores and instead focuses on ranking relationships, providing more reliable feedhack signals for the generative model.
0.8
0.8
0.8
Accuracy
0.7
Accuracy
0.7
Accuracy
0.7
0.6
0.6
0.6
0.5
0.5
0.5
0.4
Score- based Ranked-
0.4
based
Ranked-based
0.4
based
(a) GSK38+JNK3
(b) GSK33+INK3+QED
(c) CSK33-JNK3+QE--SA
(2 objectives)
(3 objectives)
(4 objectives)
Figure 9. Accuracy comparison of score-based and rank- proxy models in predicting the ranking of randomly selected molecule pairs across varying numbers objectives: (a) 2 objectives, (b) objectives, and (c) objectives.
Rank-based proxy. To demonstrate the effectiveness of the rank-based proxy, we compare the performance score-based
25

---

Offline Model-based Optimization for Real-World Molecular Discovery
and rank-based proxy models in predicting the rank of randomly selected pairs of molecules. As illustrated in Figure the rank-based model consistently outperforms the score-based model across all scenarios with varying objectives. This performance gap widens the number of objectives increases, with the rank-based model maintaining relatively high accuracy even with four objectives, while the score- score-based model's accuracy drops significantly. These findings validate the superiority of the rank-based proxy over the score-based proxy in effectively addressing the complexities of offline MOMO.
E. Additional Experiments on Multiple Proxies
Impact of Multiple Proxies with Priority
0.445
Mean Hypervolume (HV)
0.440
0.435
0.430
0.425
0.420
1
16
32
64
Number of Proxies
Figure 10. An illustration of the impact of employing multiple proxies with priority sampling in our framework. The evaluation metric is the mean hypervolume across all numbers of objectives for the MPO task under full-offline setting. The results demonstrate that the optimal configuration for our framework is four proxies, achieving the best performance before decline due redundancy.
Motivation for multiple proxies. In this section, we provide a detailed process and analysis of employing multiple proxy models within our framework. The motivation for experimenting with multiple proxies arises from observations in both offline MBO and LLM research. In offline MBO, methods employing multiple proxies-such as Ensemble Proxy, ICT, and Tri-Mentoring- -eaeorainggggeeallllly outperform single proxy methods like Grad. This finding aligns with a recent study in large language models (LLMs) (Chakraborty al., 2024), which highlights the drawbacks of using single reward model to represent human preferences. Researchers note that human preferences are inherently diverse, and single model often fails to reflect this variability, leading to biased suboptimal outcomes. To address this, they propose using multiple reward models to capture broader spectrum of preference distributions, thereby enhancing alignment with diverse human judgments.
Setup for multiple proxies. Inspired by these insights, we enhance our proxy model by incorporating ensemble learning through the use of multiple proxies. the context of LLMs, preferences reflect human sentiments, opinions, or judgments about desirable outputs. In molecular optimization, however, preference represents the relative importance or priority of each objective within the optimization process. To effectively capture this diversity of priorities, employ priority sampling for each proxy model, allowing them to prioritize objectives differently according their assigned importance. Specifically, each proxy receives weight configurations sampled from a Dirichlet distribution. enabling it to focus more on certain objectives than others. As a result, each proxy can determine which molecule a given pair is superior from its unique perspective. These individual assessments are then combined using a majority voting strategy, providing a comprehensive evaluation of molecules from multiple viewpoints to determine the overall superior molecule.
Results for multiple proxies. As demonstrated in Figure 10, the performance of our framework increases with the number of proxies, peaking at four before gradually declining thereafter. The observed decline in performance beyond four proxies can be attributed to the balance between ensemble diversity and redundancy. For an ensemble to be effective, the individual proxy models should be diverse, each providing unique insights into molecule evaluation. While adding proxy models up to a certain point enhances performance by capturing a wider range of priorities, adding too many proxies can introduce redundancy. Beyond the optimal number, additional proxies may become similar to existing ones, offering little new information and potentially amplifying common errors. addition, with large number of proxies, majority voting can
26

---

Offline Model-based Optimization for Real-World Molecular Discovery
overlook minority opinions, reducing ensemble diversity and neglecting smaller yet significant priorities. Lastly, note that all configuration whetter employing single proxy or multiple proxies-outperfae all competing methods, underscoring the effectiveness of our framework.
Analysis for multiple proxies. One might question how majority voting works with an even number of proxies, as it could lead to a tie. such cases where the proxies are evenly split in their assessments (e.g.. two proxies favor a molecule while two do not), we interpret this as an indication of uncertainty or difficulty evaluating the molecule. Rather than making a hasty decision that could misguide the optimization process, we choose to pass and skip these uncertain molecules. This approach ensures that only molecules with higher degree of consensus among the proxies influence the optimization, enhancing the reliability of the feedback signals. The results also validate that employing four proxies surpasses the performance of using three proxies. In the four-proxy setup, a molecule must receive at least three favorable votes to be considered superior, raising the confidence threshold compared to the two-out-of-thee votes required in the three- three-proxy proxy setup. The stricter criterion in the four-proxy setup leads to more reliable and accurate feedback, contributing to improved optimization performance.
F. Additional Analysis on Molecular Diversity of StitchNet
Additional diversity metries. In the manuscript, compared the diversity achieved by StitchNet with that of its data augmentation counterpart, stochastic sampling. We found that StitchNet exhibits greater diversity, which we attribute to its crossover-like mechanism that enables the generation of considerably more diverse molecules than stochastic sampling. To further investigate the diversity achieved by StitchNet, we propose the use of additional diversity metrics to provide a more comprehensive analysis from multiple perspectives. To quantify the diversity of the augmented molccules, we employed the inverse of the Tanimoto similarity (Bender & Glen, 2004). Specifically, we calculated the maximum Tanimoto similarity for each augmented molecule with respect all other augmented molecules, then averaged these values and subtracted the result from 1, which term the 'Within augmented ed' diversity metrie. In addition, we computed the maximum Tanimoto similarity between each augmented molecule and the molecules in the offline dataset, similarly subtracting this value from I to derive the 'Against offline dataset diversity metric.
distribution augmented molecules Score distribution augmented molecules
Diversity comparison sugmented molecules
8
0.70
sampling
0.6
0.5
Divestty
0.2
0.2
0.55
Stochastic sampling
StbchWet
Stochastic sampling
Schtchwe
0.50
Within ougmennted
Aguinal offiine caluset
(a) Score distribution
(b) Score distribution
(c) Diversity comparison
Figure 11. Diversity analysis augmented molecules generated by StitchNet and its data augmentation counterpart, stochastic sampling. The results demonstrate the superior capability of StitchNet in generating diverse and novel set of augmented molecules.
Additional diversity results. The results in Figure 11 (a-b) demonstrate that StitchNet produces much broader and more varied score distribution compared to stochastic sampling. This broader distribution highlights the StitchNet's capability to generate augmented molecules with higher diversity, thereby enriching the fine-tuning process for the generative model. Moreover, the additional diversity metrics further emphasize the advantages of StitchNet over stochastic sampling. As shown in Figure 11 (c), StitchNet consistently achieves higher values in both the Within augmented and Aga Against off ffflnne dataset diversity metrics. This indicates that augmented molecules generated by StitchNet not only show greater diversity among themselves but also display more novelty in comparison the molecules present in the offline dataset. Additionally, we evaluated the final molecules produced by the generative model fine-tuned with StitchNet against those fine-tuned with stochastic sampling, shown in Figure 12. The generative model fine-tuned with StitchNet outperforms its counterpart in every aspect: (a) score distribution of final molecules, (b) diversity metrics for both Wi thin augmented and Against offline dataset, and (c) diversity based on Bemis-Murcko (BM) scaffolds and Carbon Skeletons (CS) (Bemis& & Murcko, 1996).
BM scaffolds & Carbon Skeletons. BM scaffolds are an essential tool for breaking down organic molecules to identify
27

---

Offline Model-based Optimization for Real- WoaldWorld Molecular Discovery
Diversity comparison
Substructure comparison molecules
tcchlttt
Sctchhitt
Stochastic sampling
Stochack sampling
0
2500
2000
0.4
1300
0.3
1000
0.2
0.45
0.1
Stochestic
Geichhket
(a) Score distribution
(b) Diversity comparison
(c) Substructure comparison
Figure Diversity analysis final molecules produced by the generative model fine-tuned with StitchNet and with stochastic sampling. The results demonstrate that the generative model fine-tuned with StitchNet consistently achieves higher diversity and performance across all diversity metrics.
their core chemical substructures. As shown in Figure 13 (a), BM scaffolds simplify molecules by removing side chains while preserving the core substructures- such as ring systems and connecting repprssnting the molecular backbone. This approach allows for more effective quantitative assessment of structural diversity by comparing the backbones of different molecules. Another method for assessing structural diversity is through CS, which describe various configurations of carbon atoms, including straightline, branching, and ring, depicted in 13 (b). In particular, straightline skeletons consist of carbon atoms connected in linear arrangement, while branching skeletons contain side chains that extend from the main carbon chain that potentially affects the molecule's 's reactivity and interactions with biological targets. Ring skeletons are closed loops of carbon atoms, commonly found in biologically active compounds. Both BM scaffolds and CS serve as complementary methods for simplifying and categorizing molecular structures to better understand their properties and interactions. While BM scaffolds focus on the core substructures by removing side chains and functional groups, CS emphasizes the basic carbon framework of given molecule. By incorporating both approaches in our analysis, we believe we can conduct more comprehensive evaluation structural diversity across the generated molecules.
Corresponding Molecules
ScafTold
Straightline
Branching
Ring
(a) Bemis-Murcko (BM) scaffolds
(b) Carbon Skeletons (CS)
Figure 13, Visual representations of (a) the Bemis Murckoo scaffolds and (b) the Carbon Skeletons.
28

---

Offline Model-based Optimization for Real-World Molecular Discovery
G. Pre- Tre-Training Process for the Generative Model (REINVENT)
(a) Molecule generation process
C
C
and
Generative model
Benzene (CICCCCC1)
start
(b) Pre-training process for the generative model
generative model likelihood
[...]
molecules
O
(ОCIсссссι)
Maximum likelihood
o
Generative model
objective: predict next token
ZINC dataset
(CICCCCC1)
[...]
Figure (a) The generative model produces molecules SMILES format using an auto-regressive approach. (b) During the pre-training stage, molecules from the ZINC dataset used as ground truth labels. The generative model is then updated through maximum likelihood approach to maximize the probability of the correct next molecular token (component) given the preceding sequence.
In this section, we present an illustration of the molecule generation process and describe the pre-training process for the generative model. As depicted in Figure 14 (a), which exemplifies the generation of benzene molecule, the generative model produces molecules in auto-regressive manner, similar to how language models generate sentences sequentially. Specifically, the generative model produces molecules in SMILES format, where each token corresponds to atom bond. The generation process begins with an initial token, and the model predicts the subsequent token based on the previously generated sequence, continuing this process until the complete molecule is formed.
Moving on to the pre-training process for the generative model, we employ an approach analogous to the next-token prediction loss used in language model training, as shown in Figure 14 (b). Specifically, the model trained using the maximum likelihood approach, where molecules sampled from the ZINC dataset serve ground truth labels. The objective of this pre-training process is to maximize the likelihood of accurately predicting the next molecular token (component) based the preceding sequence. The cross-entropy loss is employed to measure the difference between the predicted probability distribution and the true distribution of the next token, guiding the model to learn the correct sequence of molecular components and generate chemically valid molecules.
Building upon pre-training our generative model using the ZINC dataset, we now detail specific generative model employed in our framework. As mentioned in the main manuscript, REINVENT was selected our main generative model due its widespread adoption and proven effectiveness in various molecular optimization tasks. In REINVENT, the molecule optimization process is formulated based on given scoring (reward) function. The training architecture of REINVENT comprises two distinct policy models: the prior model and the agent model. The prior model, denoted as Gref, is pre-trained reference model that encodes chemical grammar to ensure the chemical validity the generated molecules, as depicted in Figure 14 (b). The agent model Go initialized from the prior model and serves as the main policy that aims to maximize the reward score associated with the desired molecular properties, while not deviating too far from the prior model. The training objective for the agent model
Markov decision process, utilizing the RL algorithm to generate molecules
29

---

Offline Model-based Optimization for Real-World Molecular Discovery
be defined as:
Lagent( (0) = Em (-- log (m()) + log Gmr(m)) + R(m))²)
where R(m) represents the reward score for molecule m. within the offline dataset D. Note that this work addresses the offline MOMO problem, where the offline dataset comprises pairs of molecules and their corresponding property (objective) scores. Therefore, these property scores can be used as reward scores for training the agent model. To sum up, this loss function Lagent( ()) guides (m( maximize the reward R(m) while aligning with G.r((m) For detailed derivation and background of this REINVENT loss function, please refer to prior studies (Olivecrona al., 2017; Guo & Schwaller, 2024a). After completing the initial training phase the offline dataset, the agent model is fine-tuned to further enhance its performance beyond the constraints of the offine dataset. This fine-tuning process involves optimizing the agent model with stitched molecules using preference optimization techniques, as described in Equation 14 the main manuscript.
H. Preference Optimization Techniques for the Generative Model
H.1. From Initial Loss Formulation to DPO-like Loss Formulation
As mentioned in Subsection 3.3, the initial loss formulation for the generative model is follows:
Lgen (6) (m.. [log (T)))))) log (mu)) 3 [Dx (G
This loss equation consists of two key components: the first term represents the difference in log-likelihoods between generating the winning molecule Go mm)) and the losing molecule G (mi), while the second part introduces KL divergence between the current generative model Gş and the reference model Geer. Following (Tang et al., 2024), the KI. divergence term can be defined as:
DEL (GLl = E(m)-B log (m(m) (m))
Since we are focusing on pairwise comparison between winning and losing molecules, (Mow mi), it is possible to apply the KL divergence each component and simply the loss function follows:
-m ma))) 3 log Go( log Gp(ma))
( ]
Egen (Ø)
(
At this point, we can leverage the notion of the Bradley-Terry model that the log odds of one item winning over another (in our case, My over mi) can also be written as:
Go
log
G (m))
and this log-odds can be converted into probability using the sigmoid function (-). defined as:
(x)) 1te-2
To incorporate the probabilistic nature of the comparison, we can now apply the sigmoid function to a combination the two log-odds from Ge and Get as follows:
[s log (T()))) Go (iTH) (m))
log
(mr))
Finally, the initial formulation can be re-organized into the following compact DPO-like form:
Go(Mw) (m)
G Gm(m)) G (mi)
dpo (Ø) = - -E (m
log
BBog
Blog
30

---

Offline Model-based Optimization for Real-World Molecular Discovery
H.2. IPO-like Loss Formulation
Building on the methodology presented in (Tang et al., 2024), we can represent the DPO-like loss formulation in more generalized form follows:
F (3((11)) log ( log Gre((mbw)
(a(())) (m)) )
Egcn-dpo (Ø) - [m (m YB
where FF is a scalar function F R I. that map input values to scalar outputs. In the case of DPO, F is typically chosen to be the log-sigmoid function. However, DPO can encounter difficulties when preferences are deterministic. For example, the probability of mw defeating my is exactly 1, indicating deterministic preference, the difference between them becomes unbounded and approaches toward infinity such as follows:
GolMa) (
Go Ga(mu) (ie)(ma))
)
(log Ga(mw) log Go(mi) (m)
Gier(mi)) )
+00.
Assuming that B is positive real number, the term inside the log-sigmoid function becomes infinite, leading to:
( (108 log Ga(mu)) log Ga(mi) )) log oggg)+)))
log
(ml)))
Gia((T))
Since o++00) 1, it follows that:
log l+go(++o)) = log(1) 0.
Therefore, when preferences are deterministic, the loss function converges to 0 for any value of B. In other words, the regularization term 3 becomes irrelevant and does not play any role such cases.
To address these challenges, IPO introduces a stronger regularization term that penalizes models for exhibiting excessive confidence in preference margins. Specifically, IPO replaces the log-sigmoid function used in DPO with squared loss function (Tang et al., 2024). The quadratic nature of the squared loss penalizes large deviations more heavily, discouraging the model from generating extreme outputs (Rosasco al., 2004). In deterministic preference cases, the squared loss establishes the boundary to prevent the loss function from converging to 0 for any value of 3 (Azar et al., 2024).
Recall that we can express the IPO-like loss formulation as follows:
(108 (G(m))) (mu)
1 27
(gen-ipo(@)) = (Bm
(G( Gre((mw
28
As shown, the squared loss function is implemented and is explicitly positioned outside the logarithm term. Let us examine the behavior of this IPO-like loss for different values of (8. In the case of B co, the term simplifying the loss function to:
[[(oos
G (m( G ( Go(m))
Cgen-ipo (Ø) -B
Giet((ee.)))
To minimize this loss, the following conditions should ideally be met:
Ga(min) Gre((mwe) Go(mu) Grar(mt))
Thus, as 3 + oo, our current model Go converges to the reference model Gief. In contrast, as 3 + O, the term 28 - xo begins to dominate the loss function, causing the IPO-like loss to converge toward the DPO-like formulation. This suggests that the IPO-like loss exhibits distinct behavior depending on the value of 3, even in deterministic preference scenarios. In contrast, the DPO-like loss renders (3 irrelevant in such scenarios, meaning the loss remains unaffected by changes in B.
31

---

Offline Model-based Optimization for Real-Worle Molecular Discovery
I. Self-Supervised Training Process for StitchNet
Sulf-Superuised Training
Fragment molecule pairs
Stitched molecules
Molecular similarity
Fragment
Fragment molecule
Similarity measure
Stitched molecule
molecule I
mulecule
Fragmentation
StitchNet
Fragment malecule
Fragment molecule
Stitched molecule 2
Stitched
0.6
molecule
Original molecule
Original molecule
Similarity
> threshold
(score orig 0.8)
score
Fragment malecule
Fragment molecule
Stitched molecule
Seitched molecule
Stitched molecule (scoreorg 0.8)
Chemical feedback
Figure 15. An illustration of the sulf-superrvsed training process for StitchNet. An original molecule sampled from the offline dataset and decomposed into two fragment molecules using fragmentation function. These pairs of fragment molecules then fed into StitchNet to generate new stitched molecules. The molecular similarity between the stitched molecule and the original molecule measured, and if it exceeds a pre-defined threshold, the true objective score of the original molecule is leveraged chemical feedback.
In this section, we provide a detailed explanation of the ceff-supervised training process for StitchNet, which a key differentiating factor from traditional rule-based crossover operators. The importance of this process lies in its ability leverage chemical feedback, allowing StitchNet to better understand how stitched molecules are likely to exhibit objective scores when two molecules are combined. Unlike rule-based crossover operators, StitchNet is built using neural network architecture that enables it to learn from such chemical feedback.
As shown in Figure 15, we begin by sampling an original molecule from the offline dataset, each with corresponding known objective scores. We then apply fragmentation function within the crossover operator (Jensen, 2019) to decompose the original molecule into two smaller fragment molecules. There are multiple possible pairings of these fragment molecules, and we consider all viable pairs inputs StitchNet. Subsequently, StitchNet takes these pairs of fragment molecules and generates corresponding offspring stitched molecules. We then measure the molecular similarity between each stitched molecule and the original molecule. If the similarity exceeds a certain threshold (e.g.. 0.9), we consider the stitched molecule sufficiently similar to the original molecule. This high similarity allows us to leverage the known objective scores of the original molecule as an approximation for the stitched molecule's 's objective scores, effectively providing chemical feedback to StitchNet. We use this feedback to train StitchNet with the loss function specified in Equation 9, We think that this approach is reasonable based on two key assumptions. First, since the fragment molecules are derived from the original molecule, the stitched molecule is expected share similar characteristics. Second, because structurally similar molecules often exhibit similar properties (Barbosa & Horvath, 2004; Alvesalo et al., 2006; Maggiora al., 2014), assume that the stitched molecule will likely exhibit objective scores comparable to the original molecule. By ensuring that the stitched molecule is sufficiently similar to the original, we can reasonably use the original molecule's 's objective scores as an approximation for the stitched molecule's scores.
The rationale for this self-supervised training process arises from the inherent nature of the offline MOMO problem. In an online setting, would be possible to sample two molecules from the offline dataset, input them into StitchNet, generate a stitched molecule, and then query an oracle to obtain its true objective scores for chemical feedback. However, in an offline setting, additional oracle queries offline dataset, we decompose a single molecule into two fragment molecules, which are then input into StitchNet. Since the true objective scores of the stitched molecules cannot be obtained due to the unavailability of additional oracle queries, we instead leverage the objective scores the original molecule as a form of chemical feedback. This allows us to approximate the likely performance of the stitched molecule, which in turn can guide StitchNet learning how to generate stitched molecules with desirable objective scores during the molecular stitching process.
not possible. Therefore, rather than simply using two random molecules from the
32

---

Offline Model-based Optimization for Real-World Molecular Discovery
J. Priority Sampling Process for StitchNet
Priority sampling
Property
11
Property 2
Molecule A
Property Property 2
Priority
Offline Dataset
sampling
Property!
2
Property
Stitched molecule
StitchNet
Molecule B
Pareto front
Pareto front
Pareto front
Property
Property
Property
A2
Property 2
Property
Property
Figure 16. An illustration of the priority sampling process for StitchNet. The figure demonstrates how different weight configurations are sampled from the Dirichlet distribution Dir(a), guiding the selection molecular pairs from the offline dataset. For instance, Aı focuses more property while A2 emphasizes property resulting in the selection of molecules A and B. respectively. These molecules are then fed into StitchNet, which generates a novel stitched molecule with the aim of combining the desirable properties of both parent molecules. This priority sampling promotes diversity and balance in the stitched molecules, enhancing the convergence towards the Pareto front.
In this section, we visualize the priority sampling process and explain why it is beneficial for the molecular stitching process StitchNet. Consider a scenario where we are optimizing two molecular properties: property 1 and property 2, as shown Figure 16. Our goal is to sample diverse set of molecular pairs from the offline dataset; for instance, one molecule (Molecule A) exhibits characteristies more aligned with property 1, and the other molecule (Molecule B) emphasizes property 2. This diversity is crucial because StitchNet seeks to combine these molecules to create novel stitched molecule that inherits the desirable properties of each parent molecule. If we sample pairs of molecules that have similar characteristics or properties, the benefit of the molecular stitching process diminishes due to the lack of diversity. To address this, we propose using priority sampling with Dirichlet distribution to automatically generate diverse weight configurations, denoted as A. Different weight configurations indicate varying levels of importance or priority for cach property, allowing us to sample molecules from offline dataset using different perspectives or priorities.
As depicted in Figure 16, Å represents weight configuration that focuses property 2. It is important note that these weight configurations sampled from the Dirichlet distribution Dir(a). Based on these configurations, corresponding molecules are sampled from the offline dataset and fed into StitchNet molecule A and molecule B. StitchNet then generates a novel stitched molecule with the aim of possessing both favorable property 1 and property 2. In terms of the Pareto front, sampling molecules based on A corresponds to selecting molecules near the y-axis (emphasizing property 1), whereas X corresponds to molecules near the x-axis (emphasizing property 2). By performing molecular stitching via StitchNet, we aim to generate molecules that balance both properties, thereby improving convergence towards the Pareto front. Please note that weight configurations for focusing on property I and property merely example for better understanding. In practice, the Dirichlet distribution allows to automatically generate diverse combinations of weight configurations.
property while X2 emphasizes
sample diverse molecular pairs using priority sampling, as
K. Pseudo-Code
33

---

Offline Model-based Optimization for Real-World Molecular Discovery
Algorithm 1 StitchNet
Input: StitchNet Sg. Unlabelled dataset ... Offline dataset D, Crossover operation Crossover, Dirichlet distribution Dir,
Concentration constant o, Fragmentation function Cut, Similarity threshold Similarity function s sim
Output: Generated stitched molecules m
b Pre-training for StitchNet
Sample parent molecules from unlabelled chemical dataset; m, ~ Du and mj Dr, Generate offisping molecule with crossover operation; m, k- Crossover(mi, my)
Train StitchNet S resemble crossover operation; v 6- arg max P( Sy (m,, my)) Set pretrained StitchNet reference model; Siet <- Sw
b Self-supervised training for StitchNet Sample objective preference; X Dir(a))
Sample molecule and its score from offline dataset with preference; r') 2 D
Cut m, into all possible Z fragment molecule sets; (Mmai ma))] f- Cui(m)) Find the most similar offisprigg and its fragment set with original molecule msi
(ma, my < arg max im,,,, Crossover(m.ss m,x) subject to sim(.) 2
(ma..ma))
Provide chemical feedback to StitchNet while maintaining the chemical validity; Sanch (1b) - lolog S mm) + log ( + R(Merg))
eq.9;
Molecular Stitching
Sample two objective priorities: A1 Dir(a) and X2 Dir(a)
Sample parent molecules of different objective priorities; À DD and ma À D Generate novel stitched molecule using fine-tuned StitchNet; m ~ (m,, m2)
Return 77
Algorithm 2 MolStitch
Input: Pretrained Generator Grer Pretrained StitchNet See, Offline dataset D, Proxy model fo. Dirichlet distribution Dir Concentration constant a, Output: Final molecules for evaluations final
Initialize Generative model; G6 f- Gief
Initialize StitchNet; Sg Sier
Update Generative model G. with offline dataset D:
Train proxy model fo with pairwise ranking loss in eq.7; Dir(ax)
Sample objective preference;
Finetuning StitchNet S.P with preference A;
Sample objective preferences; \1, X2 Dir(a.
Sample stitched molecule mn by molecular stitching; m Sy(m,,
Determine winning and losing molecules using proxy model fo by eq.11; (m fo(m) Fine-tuning Generative model with IPO-like loss in eq.
Sample final molecules for evaluations; m minal G
Return minin
34

---

Offline Model-based Optimization for Real-World Molecular Discovery
L. Experimental Details
In this section, we provide a comprehensive overview of our experimental setup, covering (1) experimental settings and configurations, (2) a detailed description of the molecular objectives used in our study, and (3) an in-depth discussion of hyperparameters and implementation details.
I.1. Experimental Settings and Configurations
Oracle calls. In this work, we conducted two main experiments: 1) Molecular Property Optimization (MPO) task (Gao et al., 2022) and 2) docking score optimization task (Lee et al., 2023). Recall that both experiments were designed to simulate real- world constraints by restricting the number of oracle calls, which represent expensive evaluations of molecular properties. For the MPO task, the total number of oracle calls was limited to 10,000 (Gao et al., 2022). Following this guideline, we allocated 5,000 calls to construct the offline dataset and reserved the remaining 5,000 for evaluation. Specifically, we used the initial 5,000 oracle calls to build the offline dataset, which served as the training data for developing and fine-tuning the generative model during the offline optimization process. After completing offline optimization, the performance of the fine-tuned generative model was evaluated using the remaining 5,000 oracle calls on the molecules newly generated. For the docking score optimization task, the total number of oracle calls was restricted 3,000 (Lee et al., 2023). This lower allocation might be due to the longer time required for evaluating docking scores. Similar to the MPO task in concept, we allocated 1,500 oracle calls to construct the offline dataset and the remaining 1,500 to evaluate the performance of the fine-tuned generative model.
Offline dataset collection. To construct the offline datasets for both experiments, we utilized the ZINC dataset (Sterling & Irwin, 2015). which is a publicly available chemical database that provides a collection of commercially available compounds. The ZINC dataset offers a wide variety of molecular structures, providing a large chemical space to explore for potential drug candidates. Its compounds are also available in formats suitable for molecular docking, making it good resource for identifying potential compounds that may bind to biological targets. Therefore, we considered the ZINC dataset to be well-suited for both the MPO task and the docking score optimization task. It is worth noting that we also used the ZINC dataset during the pre-training stage; however, at that stage, we only utilized the molecular structures without any associated objective scores or additional information. When aiming to optimize specific molecular objectives, we needed to query the oracle to obtain the objective scores of molecules within the ZINC dataset. For the MPO task, we randomly sampled 5,000 molecules from the ZINC dataset and executed 5,000 oracle calls to evaluate their corresponding molecular objective scores, such as JNK3, GSK3,3 QED, and SA. We collected this data in the form of (molecule, object ive scores) pairs. Similarly, for the docking score optimization task, we randomly sampled 1 1,500 molecules from the ZINC dataset and performed 1,500 oracle calls to evaluate their corresponding docking scores for five proteins alongside QED and SA. These constructed offline datasets were subsequently used for offline optimization in our proposed framework as well as across all competing methods to ensure a fair comparison.
L.2. Descriptions of the Molecular Objectives
In this work, we adopted four commonly used molecular objectives- GSK33, QED, and SA- for the MPO task. For the docking score optimization task, we targeted the docking scores of five proteins-parpl. fa7, jak2, braf, and altls-alongside QED and SA. The docking scores were calculated following the experimental protocol of prior work (Guo & Schwaller, 2024b), using the normalized QuickVinal docking score (Alhossary et al., 2015). Specifically, the normalized docking score (DS) is calculated using the given equation:
DS
Normalized DS
20
where DS represents the original docking score. Detailed descriptions of each molecular objective and protein are provided below.
JNK3. JNK3 is member of the c-Jun N-terminal kinases (JNKs) family, which belongs the mitogen-activatee protein kinase (MAPK) pathway and is primarily expressed in the central nervous system (Bogoyevitch & Kobe, 2006). It plays a crucial role in mediating cellular responses to stress, including apoptosis, inflammation, and neuronal damage (Bogoyevitch & Kobe, 2006). Targeting JNK3 inhibition is one of the key molecular objectives in drug discovery because it may prevent or reduce neuronal cell death and inflammation, making a promising therapeutic target for neurodegenerative diseases such as Alzheimer's disease (Resnick & Fennell, 2004).
35

---

Offline Model-based Optimization for Real-World Molecular Discovery
GSK33. Glycogen synthase kinase beta (GSK33) is a serine/threonine protein kinase involved in various cellular processes, including glycogen metabolism, cell proliferation, differentiation, and apoptosis (Beurel al., 2015). It has gained significant attention in neurodegenerative disease research due to its role in regulating tau protein phosphorylation, amyloid precursor protein processing, and neuronal survival (Jope et al., 2007). Inhibiting GSK33 is considered a vital molecular objective in drug discovery, as it could modulate these pathological processes and potentially slow or prevent the progression of neurodegenerative diseases (Jope et al., 2007).
QED. Quantitative Estimate of Drug-likeneee (QED) metric widely used in molecular optimization to evaluate the drug-likeness of a molecule (Bickerton al., 2012). It consists of several physicochemical properties, including molecular weight, lipophilicity (logP), topological polar surface area (TPSA), the number of hydrogen bond donors and acceptors, and the count of aromatic rings and rotatable bonds (Guan et al., 2019). It provides a score ranging from O to 1, with higher scores indicating molecules that are more likely to have favorable drug-like properties.
SA. Synthetic Accessibility (SA) is a metric used in molecular optimization to assess the ease with which a molecule can be synthesized a laboratory setting (Ertl& & Schuffenhauer, 2009). considers various structural features that influence synthesis complexity, such as the presence of complex ring systems, functional groups, stereocenters, and the overall size and branching of the molecule (Ertl& & Schuffenhaue, 2009). The SA score ranges from to 10, with lower scores indicating higher synthetic feasibility. In this work, transform the SA score into the normalized SA score, following prior studies (Lee et al., 2023; Guo & Schwaller, 2024b), to formulate it as maximization objective. Specifically, the normalized SA score is given by the following equation: Normalized SA = 10 SA
9
This adjustment ensures that higher normalized SA scores correspond to molecules that are easier to synthesize, within the score range of 0 to 1.
parpl. Poly (ADP-ribose) polymerase (parp1) is a protein enzyme that plays a crucial role in DNA damage detection and repair (Rouleau et al., 2010). is involved in various cellular processes, including chromatin remodeling, transcriptional regulation, and cell death signaling (Ray Chaudhuri & Nussenzweig, 2017). In recent years, dysregulation of parpl activity has been linked to several neurodegenerative diseases, such as Parkinson's 's disease, where excessive activation of parp1 can lead to neuronal death through a process known as parthanatos (Liu et al., 2022). Consequently, targeting parp1 has become a key molecular objective in drug discovery, not only for cancer treatment but also for developing neurotherapeutics aimed at preventing neuronal loss (Zhang et al., 2023).
fa7. Coagulation factor VII (fa7), also known as proconvertin, is vital protein in the blood coagulation pathway (Hall et al., 1964). It plays a crucial role in initiating the clotting process by activating factor X in the presence of tissue factor (TF), leading to the conversion of prothrombin to thrombin and ultimately forming blood clot (Eigenbrot, 2002). Targeting fa7 represents another key molecular objective in drug discovery, particularly for managing thrombotic and cardiovascular diseases. Specifically, inhibitors of fa7 are being explored as potential anticoagulants to prevent and treat conditions such as deep vein thrombosis, embolism, and stroke (Robinson et al., 2010).
jak2. Janus kinase 2 (jak2) is a non-receptor tyrosine kinase that plays a critical role in the signaling pathways of various cytokines (Yamaoka et al., 2004). It is involved in various cellular processes, including cell growth, differentiation, and immune function (Seavey & Dobrzanski, 2012). In drug discovery, jak2 has gained attention due to its association with mycloproliferative neoplasms and other hematological malignancies (Senkevitch & Durum, 2017). Inhibiting jak2 is considered a key molecular objective, as it can potentially provide therapeutic benefits in inflammatory and autoimmune disorders (Seavey& & Dobrzanski, 2012).
braf. B-Raf proto-oncogene (braf) encodes a serine/threonine kinase that is part of the MAPK/ERK signaling pathway, which plays a crucial role in regulating cell growth and migration during various cellular processes (González-Gonzzález et al., 2020). Mutations in the braf gene are commonly found in various cancers, including melanoma, colorectal cancer, and thyroid cancer (Atiqur Rahman et al., 2014). Therefore, targeting the braf can be critical therapeutic objective in oncology to target these cancer-specific mutations and halt the progression of the disease (Sanz-Garcia et al., 2017).
5htlb. p-yyoooyttyplammnne receptor IB (5htlb) is a G protein-couplet receptor that binds serotonin (Launay et al., 2002). It is widely expressed in the central nervous system and plays important roles in regulating neurotransmitter release, neuronal firing, mood, and appetite (Fink & Göthert, 2007). Shtlb has emerged as an important molecular target in drug discovery for neurological and psychiatric disorders, particularly in the treatment of migraine and depression (Giniatullin, 2022).
36

---

Offline Model-based Optimization for Real-World Molecular Discovery
L.3. Hyperparameters and Implementation Details
Implementation of the generative model. We closely followed the architecture settings for REINVENT as described in the PMO benchmark (Gao et al., 2022), while the settings for GFlowNets were based GeneticGFN (Kim et al., 2024a), and those for Mamba were taken from Saturn (Guo & Schwaller, 2024b). Since all of these generative models were originally designed for an online setting, we made necessary adjustments to the number of molecule updates and the experience replay to adapt them for our offline settings. The final hyperparameters for the generative models were primarily determined based on the performance of REINVENT, which served as our backbone generative model, and are detailed in Table 17.
Stabilizing GFlowNets. During the training of GFlowNets, we encountered instability with the original setting of the 1og2 parameter, which plays crucial role in trajectory balancing and needs be adjusted according to specific settings (Malkin et al., 2022). To be more specific, it was initially set to high value (logZ = 5.0) with learning rate of 0.1, as specified in GeneticGFN, To stabilize the training process, reduced the logZ value to 0.001 and aligned the learning rate with that of the generative model (from 0.1 to 0.0005). This adjustment resulted in more stable training and significantly improved performance. Additionally, during preference optimization, while both REINVENT and Mamha require only the generative model's '' likelihood as input, we recommend using the sum of likelihood and log2 for GFlowNets in order to further improve performance.
Hyperparameters for StitchNet. Recall that StitchNet combines two parent molecules input and generates stitched molecules in an auto-regressive manner. Therefore, it operates by computing the hidden dimensions h1 and h2 of two parent molecules my and 1/2, respectively, and then averaging these hidden dimensions as Aسي StitchNet is built upon the REINVENT architecture. During the self-supervised training process for StitchNet, we applied a similarity threshold = 0.8 between the original molecules and the stitched molecules. During the molecular stitching process, StitchNet combines two parent molecules, each sampled with different weight configurations through priority sampling. The resulting stitched molecules are stored in a buffer. Once the buffer is full, two molecules are randomly sampled to create non-overlapping pairs. These pairs are then evaluated by the proxy model to identify the winning and losing molecules. Subsequently, the IPO-like loss is applied to increase the likelihood of generating winning molecules while reducing the likelihood of generating losing molecules. The hyperparameter settings for StitchNet are summarized in Table 18.
Table 17. The hyperparameter settings for generative models in MolStitch framework.
REINVENT
GFlowNets
Mamba
Batch size
200
Batch size
200
Batch size
200
Embedding dimension
128
Emhedding dimension
128
Emhedding dimension
256
Hidden dimension
512
Hidden dimension
512
Hidden dimension
256
Number of layers
3
Number of layers
3
Number of layers
12
Sigma
500
Sigma
500
Sigma
500
Experience replay size
300
Experience replay size
300
Experience replay size
300
Augmentation round
Augmentation round
8
Augmentation round
8
Batch update
Batch update
2
Batch update
Learning rate
5e-04
Learning rate
5e-04
Learning rate
Sc-04
log2
0.001
Table 18. The hyperparameter settings for StitchNet.
Molecular stitching
a for priority sampling
1.0
Number of stitch rounds
16
Stitched molecules per stitch round
250
Population pool
1000
Temperature 3 for IPO
0.2
37

---

Offline Model-based Optimization for Real-World Molecular Discovery
M. Extended Related Work
M.1. Generative Models for Molecular Discovery
The rapid advancement of generative models has profoundly impacted various fields, including computer vision (Croitoru et al., 2023), natural language processing (Chang et al., 2024), and audio signal processing (Deshmukh et al., 2023). This progress has extended molecular discovery (Anstine & Isayev, 2023; Son et al., 2024) and drug discovery (Jiménez-Luna et al., 2021; Shin et al., 2025b), where generative models have proven effective in designing and optimizing molecules towards promising regions of the chemical space. Various types of generative models have been proposed, such follows:
Genetic algorithms. Inspired by the principles of natural evolution, genetic algorithms (GAs) also known as evolutionary search methods (Shin et al., 20ana)-maintain a population of candidate solutions that are iteratively refined based on a predefined fitness function. These algorithms enhance population quality over successive generations by applying key operations such as selection, crossover. mutation, and replacement. In the domain of molecular discovery, GraphGA (Jensen, 2019) has shown significant promise in generating high-quality molecular structures using GA-based techniques.
Sampling- based methods. These methods leverage advanced sampling techniques to draw samples from distributions that are likely to yield desirable molecular properties. MARS (Xie et al., 2021a) is notable example that employs Markov Chain Monte Carlo (MCMC) sampling to efficiently search for high-quality molecules. By focusing on probabilistic sampling, these methods can explore the chemical space more efficiently than deterministic approaches.
Reinforcement learning (RL.. RL.-based methods formulate the molecule generation process as a Markov decision process, allowing an RL agent to interact with chemical environment to construct molecular structures in an autoregressive manner. A prominent example is REINVENT (Olivecrona et al., 2017), which utilizes GRU model (Chung et al., 2014) its RL agent to generate molecules in SMILES format. REINVENT has been acknowledged as one of the best models for various molecular property optimization tasks, showcasing the effectiveness of RL-based methods. More specifically, the strength of RL-based methods lies in its effective use of trial-and-error to navigate the environment (Angermueller et al., 2019; Shin et al., 2022; 2024a), allowing it to discover promising solutions and achieve robust performance across diverse scenarios. Following its success, several variants have been proposed to enhance its capabilities. One line of research focuses on improving the underlying neural architecture by replacing the GRU with either transformer (He et al., 2022) or Mamba (Gu& & Dao, 2023; Lee et al., 2025). Another approach incorporates data augmentation techniques to boost sample efficiency, leading to methods like Augmented Memory (Guo & Schwaller, 2024a), which achieved new state-of-ther -hetttt-rt performance.
GelowNets. While RL-based methods have shown effectiveness, they often struggle with maintaining diversity in the generated molecules due to a tendency to exploit a single promising direction. GelowNets (Jain et al., 2022; M Ghari et al., 2024; Son et al., 2025) aims to address this limitation by emphasizing probabilistic sampling over reward maximization, inherently promoting diversity in the generated molecules. As a result, GFlowNets have gained popularity in multi-objective molecular optimization tasks, where generating diverse set of high-quality molecules across multiple objectives is crucial.
M.2. Multi-Objective Molecular Optimization
The multi-objective molecular optimization (MOMO) problem differs from single-ohjective optimization by requiring the simultaneous optimization of multiple molecular properties, which often conflict with one another. In the context of the MOMO problem, identifying a single solution that optimally satisfies all objectives is generally infeasible. Instead, the goal shifts to discovering diverse set of Pareto optimal molecules, where improving one objective may lead to trade-offs in others. To tackle multiple objectives, several studies have integrated Bayesian optimization (BO) within their molecular optimization frameworks. For instance, GPBO (Tripp et al., 2021) incorporate BO into GraphGA resulting in enhanced sample efficiency. In a similar approach, LamBO (Stanton et al., 2022) applics BO alongside denoising autoencoders to address the multi-objective biological sequence design problem. Other studies have employed scalarization, which simplifies the multi-objective problem by converting multiple objectives into a single scalar objective function (Gunantara, 2018). This scalarization is typically achieved by combining the objectives using weighted sum or other aggregation techniques (Marler & Arora, 2010; Deb et al., 2016). Scalarization offers simplicity and ease of implementation, making it a popular choice for its scalability and computational efficiency (Cho et al., 2017). In the context of the MOMO problem, MIMOSA (Fu et al., 2021) utilizes linear scalarization to efficiently manage the complexity of multiple objectives, while demonstrating strong performance and scalability. Similarly, MARS (Xie et al., 2021a) applies scalarization to effectively handle up to four molecular objectives, further showcasing the potential of scalarization in the MOMO problem. However, scalarization presents challenges in selecting appropriate weights. Users must assign weights to each objective to reflect its relative
38

---

Offline Model-based Optimization for Real-World Molecular Discovery
importance, a process that is often sensitive and subjective (Royer et al., 2023). Incorrect or biased weight selection may fail to accurately represent true preferences, potentially resulting in suboptimal solutions (Zhang & Golovin, 2020). In our study, we also employ the scalarization approach due to its widespread adoption and practical advantages (Fromer & Coley, 2023), However, to mitigate the limitations associated with subjective weight selection, we introduce priority sampling using the Dirichlet distribution to generate a diverse set of weight configurations. This enables our StitchNet to operate on a wide variety of molecular pairs, each representing a different balance of multiple objectives.
M.3. Offline Model-based Optimization
As mentioned earlier in the main manuscript, one of the most promising approaches for addressing the offline MOMO problem is offline model-based optimization (MBO) (Trabucco et al., 2022). The goal of offline MBO is to optimize the objective function using a pre-collected offline dataset, without the ability to acquire new data during the optimization process. In this approach, the proxy (surrogate) model- as Gaussian processes, random forests, or neural networks is trained on the offline dataset to approximate the objective function (Dao et al., 2024). This proxy model is then used to predict objective scores for new inputs, guiding the optimization algorithm in finding inputs that maximize the predicted objective scores. The most straightforward approach in offline MBO is to use a differentiable vanilla proxy model and apply gradient ascent to find optimal inputs. However, this approach may face limitations such as increased inaccuracies and a higher risk of overfitting. To address these limitations, various recent studies have been proposed.
Improving the proxy model. One line of research focuses on enhancing the accuracy and robustness of the proxy model to better handle high -igh-imensiooal and complex objective functions. Some studies (Trabucco et al., 2021; Qi et al., 2022) enforce constraints mitigate overfitting and address distributional shifts caused by out-of-distribution (OOD) inputs, while another study (Yu et al., 2021) enhances the generalization capabilities of the proxy model by employing local smoothness prior. Additionally, more sophisticated methods such as gradient matching (Hoang et al., 2024) and policy-guided gradient search (Chemingui et al., 2024) have been proposed to improve the proxy model and overall performance.
Ensemble learning. To leverage the benefits of ensemble learning, several studies (Trabucco et al., 2022; Yuan et al., 2023; Chen et al., 2023a) have proposed to utilize multiple proxy models to combine their predictions, thereby enhancing the robustness and reliability of the optimization process. Building on the effectiveness of ensemble learning, we conducted additional experiments utilizing multiple proxy models, described in Appendix E. The results reveal that increasing the number of proxies improves the performance of our framework, underscoring the robustness and reliability that ensemble methods bring to the optimization process.
Improving optimization algorithms. Another line of research concentrates on improving the optimization algorithms used within the offline MBO framework. For example, the bidirectional learning technique (Chen et al., 2022; 2023b) has been introduced to utilize both forward and backward mappings to generate input configurations that are likely to produce optimal outputs while adhering the data distribution of the offline dataset. Furthermore, the synthetic pretraining technique (Nguyen et al., 2023) has been proposed to enhance sample efficiency and overall performance by integrating unsupervised learning with in-context pretraining. Additionally, the bootstrapping technique (Kim et al., 2023) has been developed to enhance the optimization process by iteratively augmenting the offline dataset with self self-generated data.
M.4. Preference Optimization in Generative Models
In recent years, preference optimization has gained significant attention, particularly with the rise of large language models and generative models (Tang et al., 2024). As these models grow more powerful and are deployed into real-world applications, the need to align their outputs with human expectations becomes increasingly important. Preference optimization enables models to better align with human standards in subjective areas such as sentiment, creativity, and ethical considerations.
Reinforcement Learning from Human Feedback (RLHF). A leading and widely adopted method for incorporating human preferences into model training is RLHF (Ouyang al., 2022). By embedding human feedback within an RL. framework, RLHF allows models to generate higher-quality content that aligns more closely with human judgments. Notable implementations like OpenAl's ChatGPT (Achiam et al., 2023) have demonstrated significant performance improvements through RLHF, highlighting its potential in fine-tuning models. This success has driven further research into more streamlined approaches that aim to simplify the incorporation of human preferences.
Direct Preference Optimization (DPO). DPO (Rafailov et al., 2023) is a recent method that moves away from RL and focuses directly on optimizing for human preferences without the need for reward modeling. It operates by directly
39

---

Offline Model-based Optimization for Real-World Molecular Discovery
training on human preference pairs, enabling the model to generate outputs that are consistently favored over less preferred alternatives. This approach is considered more straightforward and potentially more stable than RLHF, as it bypasses the complexities associated with RL training. However, DPO has exhibited limitations, particularly in scenarios involving deterministic preferences, due to its relatively weak regularization mechanisms.
Identity Preference Optimization (IPO). IPO (Azar et al., 2024) is more recent method that builds on DPO by introducing enhancements to address its limitations and offering a more theoretically sound framework. Specifically, IPO incorporates a stronger regularization term that penalizes models for excessive confidence in preference margins. This is achieved by replacing the log-sigmoid function used in DPO with a squared loss function. The stronger regularization term in IPO aims to balance adaptation the preference dataset while maintaining generalization capabilities, which is crucial for model performance on out-of-distribttion (OOD) data. While IPO offers theoretical improvements over DPO, empirical results have been mixed. Some studies report IPO performing on par with or slightly better than DPO (Pal et al., 2024; Calandriello et al., 2024), while others observe diminished performance in certain settings (Hu et al., 2024b).
Preference optimization in molecular discovery. In large language models, preference typically reflects human sentiments, opinions, or judgments about what constitutes desirable output. On the other hand, in the field of molecular discovery, preference represents the relative importance of each objective within the optimization process. When the generative model is tasked with optimizing several conflicting objectives, preference guides the optimization process by specifying how much weight or priority each objective should be given. For example, if a researcher wants to prioritize potency over safety, their preferences would assign more importance to optimizing potency. Conversely, if safety is more critical, the preference would shift toward that objective. Recently, preference optimization has been widely adopted in structure- structure-based drug design to align the pre-trained generative model with preferred functional properties (Liu et al., 2024a; Gu et al., 2024). Our work also focuses on optimizing molecules with desired properties. However, unlike recent studies (Gu et al., 2024) that primarily use DPO and rely on existing preference datasets, our approach differs in several key ways. We explore a variety of preference optimization molecular optimization problem. More importantly, we generate new preference dataset using our StitchNet model, which creates novel stitched molecules with desirable properties from pairs of existing molecules. In other words, rather than depending solely on existing datasets for preferences, we construct a separate proxy model and use StitchNet to build a tailored preference dataset, leveraging existing molecules to further enhance the optimization process. Additionally, we extend our approach to a semi-offline setting- direction that recent studies have not explored yet. In this setting, we utilize a limited number of online evaluations by periodically querying an oracle function to assess molecules in large batches. This extension allows us explore ways of further enhancing the optimization process by integrating new evaluation data.
innlluuinnn RLHF, DPO, and IPO-an apply them to the offline multi-objective
N. Competing Methods Details
In this section, we present a comprehensive review of the competing methods, highlighting their core principles, methodolo- gies, and their comparative position relative to our proposed framework. Before delving into the details, we first aim to explain how molecular optimization methods, such as REINVENT (Olivecrona et al., 2017), are adapted to offline settings. While we use REINVENT as an example, this approach applies to all competing molecular optimization methods. In online settings, REINVENT actively generates molecules, queries the oracle to obtain objective scores as rewards, and updates the log-likelihood of generating those molecules based on the feedback. In contrast, in offline settings, it relies on a pre-existing offline dataset containing pairs of molecules and their corresponding objective scores, rather than actively generating and evaluating new molecules through oracle queries. This offline dataset becomes the sole source of information for training and optimizing the generative model. In this context, REINVENT computes the log-likelihood of a molecule and utilizes the corresponding objective scores from the offline dataset as rewards, updating itself in a supervised manner. This adaptation enables REINVENT to operate in offline settings, leveraging the available offline data to refine its generative capabilities.
REINVENT (Olivecrona et al., 2017) is a reinforcement learning (RL) approach designed for molecular generation, where an agent interacts with its environment to create molecules. This approach autoregressively generates molecules as SMILES strings, with each new element (token) in the sequence building upon the previously generated elements. Note that this generation process is guided by a pre-trained model that enforces chemical grammar rules, ensuring the validity of the generated molecules. REINVENT has demonstrated superior performance in molecular optimization tasks, as highlighted by the PMO benchmark. This remarkable performance has led numerous follow-up studies to adopt REINVENT as their backbone generative model. Following this established trend, we have also integrated REINVENT as our backbone model to take advantage of its proven effectiveness in various molecular optimization
40

---

Offline Model-based Optimization for Real-World Molecular Discovery
tasks. As a competing method in our smudy, REINVENT serves as a baseline, as it is trained exclusively on the offline dataset without applying further offline MBO techniques. This straightforward approach positions it reference point for evaluating the effectiveness of various MBO techniques, which leverage proxy models to fine-tune the generative model beyond the constraints of the offline dataset.
Augmented Memory (AugMem) (Guo & Schwaller, 2024a) builds upon the REINVENT method by incorporating molecular data augmentation techniques and experience replay to enhance performance. The authors report that AugMem has achieved state-of-the-art results on the PMO benchmark, showcasing its effectiveness in molecular optimization tasks. In the context of offline optimization, offline MBO techniques typically use proxy models to guide the generation of synthetic data. This process involves the generative model producing new data points, which are then evaluated by the proxy model. The resulting augmented dataset allows the generative model to explore beyond the initial offline dataset. AugMem, in contrast, introduces a different approach to data augmentation specifically designed for molecular generation. By implementing AugMem in our study, we establish a valuable reference point for comparing specialized molecular data augmentation techniques against the proxy model-guided approaches used in conventional offline MBO.
GraphGA (Jensen, 2019) is method based genetic algorithms that generates molecules by evolving population through repeated cycles of selection, crossover, and mutation, all driven by fitness function. GraphGA utilizes domain knowledge from the chemical experts to develop effective mutation and crossover strategies that facilitate an efficient exploration of molecular space, In our study, GraphGA serves as a key reference for implementing rule-based crossover operations, which we have also incorporated into our framework.
GenetieGFN (Kim et al., 2024a) integrates genetic algorithms into the GFlowNets model for molecular generation. Specifically, this method leverages domain-speciff genetic operators to efficiently explore the chemical space, en- abling the generative model to implicitly acquire relevant domain knowledge. Consequently, the generative model's performance is enhanced through the strategic guidance provided by the genetic algorithm. The authors also highlight a complementary relationship between the two components: the genetic algorithm enhances GFlowNets' capacity for effective exploitation, while GFlowNets, in turn, increases the population diversity for the genetic algorithm. In our study, GeneticGFN serves as a crucial reference point for evaluating the effectiveness of genetic algorithms in the offline MOMO problem. Specifically, it allows us assess the advantages gained from incorporating domain-specific knowledge through genetic operators in this context.
Saturn (Guo & Schwaller, 2024b) builds upon the core mechanism of REINVENT while introducing significant architectural improvements. While REINVENT employs a GRU architecture. Saturn replaces with the more powerful Mamba architecture. This substitution is motivated by Mamba's 's potentially greater capacity for modeling complex molecular structures more effectively. Furthermore, Saturn incorporates genetic algorithms into its Mamba-bart based model, drawing parallels to GeneticGFN's approach. This integration allows Saturn to leverage domain-specific genetic operators, potentially enhancing its ability to navigate the chemical space effectively. In our study, Saturn serves as a valuable reference point for two key aspects: first, it demonstrates the application of the Mamba architecture in molecular optimization tasks, and second, it provides insights into the benefits of incorporating domain-specific genetic operators in the context of offline MOMO.
Grad (Zinkevich, 2003) represents the most straightforward offline MBO approach for tackling the offline MOMO problem. In particular, it employs a vanilla proxy model that directly approximates the true objective scores, training this proxy on the offline dataset. To address the generative aspect of the offline MOMO problem, Grad utilizes REINVENT as its backbone generative model, the same approach used in our proposed framework. This choice is consistently applied across all offline MBO-based competing methods to ensure a fair comparison. After training the vanilla proxy model, Grad fine-tunes the generative model using gradient ascent with respect to the trained vanilla proxy model's predictions. In our study, Grad serves as a crucial reference point as demonstrates the basic application of offline MBO in the context of offline MOMO. Specifically. Grad enables us to investigate whether a vanilla proxy model is sufficient for this task, or if more sophisticated approaches are necessary for meaningful improvements in the offline MOMO problem.
COMs (Trabucco et al., 2021) represents a more sophisticated offline MBO approach. Unlike Grad's vanilla proxy model, COMs employs adversarial learning to encourage the proxy model to provide conservative estimates of the true objective functions. This method establishes lower bounds on the objective estimates, which are then used during the
41

---

Offline Model-based Optimization for Real-World Molecular Discovery
offline optimization process. By doing so, COMs aims to prevent erroneous overestimation caused by distributional shift, common challenge in various offline optimization scenarios. In our study, COMs enables to investigate whether these sophisticated methods offer significant improvements in the context of offline MOMO.
IOM (Qi et al., 2022) considers offline MBO from a domain adaptation perspective. This method aims to train a proxy model that can accurately predict true objective scores ("target domain') when trained solely on the given offline dataset (' ('souree domain'). To achieve this, IOM introduces invariant representation learning, which enforces alignment between the learned distribution of the offline dataset and the distribution of optimized decisions. In our study, IOM serves as reference point similar to COMs, enabling us to evaluate the effectiveness of invariant representation learning in addressing distributional shifts and enhancing performance in the offline MOMO problem.
RoMA (Yu al., 2021) also addresses the challenge of overestimation issues when approximating true objective scores. To mitigate this issue, RoMA proposes robust model adaptation by incorporating local smoothness prior as regularizer. This regularizer aims to enforce a flat loss landscape, thereby enhancing the proxy model's generalization capabilities and ensuring stable training. In our study, RoMA serves as a reference point, similar to COMs and IOM, allowing us to assess the effectiveness of using regularization techniques to improve robustness and performance in the offline MOMO problem.
Ensemble proxy (Trabucco et al., 2022) takes different offline MBO approach by leveraging multiple proxy models through ensemble learning. This approach addresses the limitations of a single proxy model, which can be prone to overfitting issues. Ensemble proxy uses multiple proxies with different initializations and averages their predictions to approximate true objective scores. In our study, Ensemble proxy serves as a reference point, enabling us to evaluate the effectiveness of ensemble learning in the offline MOMO problem and assess whether the potential performance gains justify the increased computational cost associated with using multiple proxy models.
ICT (Yuan et al., 2023) utilizes multiple proxies, similar to Ensemble proxy, but enhances the approach through a co-teaching process. This process facilitates information exchange between proxies and encourages knowledge transfer. Additionally, ICT incorporates a neetallerninggbasee sample reweighting mechanism that iteratively updates the importance weights of samples to mitigate potential inaccuracies in pseudo-labels. In our study, ICT serves as a reference point, enabling us to evaluate the effectiveness of advanced ensemble techniques, such co-teaching and meta-learning, in the offline MOMO problem.
Tri-Mentoring (Chen et al., 2023a) is closely related to ICT, utilizing multiple proxies and facilitating learning between them through a mentoring process. However, Meitornnningg shifts focus to generating pairwise comparison labels rather than directly approximating objective scores. Instead of averaging predictions, it employs majority voting to combine decisions from each proxy model. In our study, Tri- Tri-Mentoring serves as a crucial reference point, enabling us to evaluate the effectiveness of using the rank-based proxy over the score-based proxy, aligning closely with the approach of our proxy model.
BIB (Chen et al., 2023b) employs a bidirectional learning approach that utilizes both forward and backward mappings to generate input configurations likely to produce optimal outputs, while conforming the data distribution of the offline dataset. BIB constructs its proxy model using pre-trained language model and applies deep linearization scheme to derive closed-form loss function. It is recognized as one of the best models for tackling the offline biological sequence design problem. In our study, BIB serves as a reference point to evaluate how well a high-performing method designed for offline biological sequence design performs in the offline MOMO problem.
BootGen (Kim et al., 2023) employs bootstrapping technique to enhance the optimization process by iteratively augmenting the offline dataset with self-generated data, using the proxy model as pseudo-labeles The goal is to align and refine the generative model through iterative training, where high-quality samples are added to the augmented dataset based on the proxy model's 's guidance. BootGen is also recognized as one of the best models for offline biological sequence design. In our study, BootGen serves as reference point to evaluate the effectiveness of the bootstrapping technique in offline optimization, and, similar to BIB, to assess how well a tigh-performing method designed for offline biological sequence design can be adapted tackle the offline MOMO problem.
RaM (Tan et al., 2025) identifies the limitations of mean squared error loss for offline MBO and instead advocates for a ranking-basee based model. By employing learning-to- to-nnkk (LTR) techniques, RaM prioritizes promising candidates based on their relative ordering rather than absolute scores, achieving otate-of-the-art performance across various offline
42

---

Offline Model-based Optimization for Real-World Molecular Discovery
MBO tasks. While RaM's ranking-based model shares conceptual similarities with the rank-based proxy mechanism in framework, work extends beyond RaM's focus by integrating the rank-based proxy one component within more comprehensive framework. Specifically, our MolStitch framework combines molecular stitching, rank-based proxy, priority sampling, and preference optimization to tackle the offline MOMO problem in real-world molecular discovery. The novelty of our work lies not in the rank-based proxy itself but developing a unified framework that leverages this component alongside others. Additionally, we enhance the rank-based proxy through experiments that incorporate multiple proxies to capture diverse priorities. Using priority sampling, each proxy model emphasizes different objectives based on their assigned importance. Our findings indicate that increasing the number of rank-base proxies improves performance, but only up certain threshold, revealing nuanced interplay between model diversity and performance. As result, our study offers more comprehensive exploration of rank-based proxy. In our study, RaM serves as critical reference point for evaluating the effectiveness of ranking- bhaee models, highlighting the advantages of these methods in offline MBO, and offering valuable insights into their broader applicability.
O. Details on Evaluation Metrics
This section provides an overview of the evaluation metrics used in this study: the hypervolume (HV) indicator (Zitzler et al., 2003) and the R2 indicator (Brockhoff et al., 2012). Both metrics are widely employed multi-objective optimization due to their effectiveness in evaluating solution quality across conflicting objectives. The HV indicator quantifies the volume of the objective space dominated by the Pareto front relative to reference point, reflecting convergence and diversity. In contrast, the R2 indicator measures how well the Pareto front aligns with a set of reference directions, assessing solution distribution. Using both metrics together provides complementary insights into the performance of optimization algorithms and the exploration of trade-offs among objectives.
0.1. Hypervolume Indicator
Hypervolume Indicator
Objective B
Hypervolume (HV)
Objective A
Figure 17. Visualization of the hypervolume (HV) indicator 2D space, where the HV corresponds the volume of the shaded region.
The HV indicator denoted as Ir. measures the volume in the objective space that is dominated by the Pareto front derived from the optimization algorithm. To be more specific, the HV indicator is defined as the volume the objective space that dominated by set of solutions X' relative to a reference point =". Of note, the reference point 2" is chosen such that is dominated by all solutions in X, representing the worst acceptable value for each objective. Mathematically, the HV can be expressed using the Lebesgue integral follows:
/ Taaraan for some some ZEX) dz",
In(X,=))
where is the indicator function that equals to if the reference point z" E RD dominated by at least one solution x E X, ie., x" < x for some x E X, and 0 otherwise. This formulation essentially measures the volume of the region in the objective space that dominated by the solutions in X' and bounded above by the reference point z". Alternatively, the HV
43

---

Offline Model-based Optimization for Real-World Molecular Discovery
can be calculated more practically follows:
U |a,, 11, AEE
In (x) Vol
where [a,=="] denotes the hyperrectangle with lower corner a and upper corner =². This representation provides a more intuitive understanding of the HV indicator directly corresponds the union hyperrectangles formed by each solution in X' with respect to 2". In nutshell, the HV indicator quantifies the size of the objective space that is simultaneously dominated by all solutions in X' and is within the bounds defined by z". A larger HV value indicates a more preferable set of solutions, as it implies that a greater portion of the objective space is covered by the set X.
To provide a clear understanding, we visualized HV as shown in Figure 17, where the blue points represent a Pareto front composed of non-dominated solutions. Then the HV defined as a measure of the region in the objective space that dominated by the Pareto front and bounded by reference point. In this study, as we have normalized all objective values between 0 and 1, we set the reference point as the origin (e.g., (0, 0) for two-dimensional space, 0,,0,) for three-dimensional space, and so on) in each respective dimensional space.
O.2. R2 Indicator
The R2 indicator (Brockhoff al., 2012) a set-based performance metric used in multi-objective optimization to evaluate the quality of set of solutions X' in approximating the true Pareto front. Unlike the HV indicator, which measures the volume of the dominated region, the R2 indicator uses set of predefined weight vectors to assess how well the solutions in X' represent various trade-offs among objectives. It is defined as the maximum of the worst-case weighted distances between the solutions in X' and an ideal or utopian point. A lower R2 value indicates better performance, as it signifies that the solutions in X' are closer to the ideal point for all considered weight vectors.
Mathematically, let W be set of weight vectors (W1, .2...., ...,...,,W .m,) where wi 2 and Σ W/ = 1, representing different priorities for the objectives. The R2 indicator, denoted R2 W), can be defined
R2 W) max min WEW XEX
Σ wy -T((() -(((x)-
where [x) is the value of the i-th objective for the solution x, and f((x) is the value of the i-th objective for the ideal utopian point (typically the maximum achievable value for maximization problems). This formulation calculates the deviation of the solution set X from the ideal point for each weight vector w and then takes the maximum of these deviations across all weight vectors in W. The use of the maximum operator ensures that the R2 indicator focuses on the worst-case scenario for any given weight vector, reflecting the least favorable trade-off among objectives that the solution set X' can achieve. A lower R2 value means that X' is closer to the ideal point across all weight vectors, indicating a better approximation the Pareto front.
In summary, the R2 indicator quantifies the worst-case performance of set of solutions X' in terms of their proximity an ideal point for given set of weight vectors W. A lower R2 value is better as it indicates closer approximation the ideal performance across all weight vectors.
44

---

Offline Model-based Optimization for Real-World Molecular Discovery
P. Further Investigations of StitchNet within Our MolStitch Framework
P.I. Quantitative Assessment of StitchNet's Ability to Learn Crossover Operations
Table 19. Assigned Scores and overall similarity between StitchNet and Crossover operator
High Scoring
Middle Scoring
Low Scoring
Similarity
Assigned Score
43%
31%
26%
0.644
In this section, we present the quantitative results evaluating how effectively StitchNet learns the crossover operation. To assess this, we generated 300 offspring molecules using rule-based crossover operations, and 100 molecules using StitchNet with the same parent molecule pairs. Then, the 300 molecules from rule-based erossover were categorized into three groups based on their mean target objective scores (GSK3)+IN3++EDD-SA) high-scoring, middle-scoring, and low-scoring, For each group, we calculated the mean Tanimoto similarity score with the 100 molecules generated by StitchNet. Each StinchNet-generateed molecule was then assigned to the group with which it exhibited the highest similarity score.
The results, presented in Table 19, demonstrate that the overall similarity scores are reasonable, suggesting that StitchNet effectively learns crossover operations through its unsupervised pre-training process. Notably, molecules generated by StitchNet were most frequently assigned to the top-scoring group, with the lowest assignment to the low-scoring group. This outcome highlights the advantages of StitchNet's self-supe training, which integrates chemical feedback to guide the generation of stitched molecules with desirable objective scores. As result, StitchNet can perform crossover operations in way that preferentially generates offspring molecules with higher objective scores.
P.2. Effectiveness and Contribution of StitchNet in Comparison to Existing Offline Dataset
Table 20, Overall improvement in objective scores when comparing stitched molecules against existing molecules in the offline dataset.
QED
SA
JNK3
GSK333
Improvement
-5.79%
-3.15%
+16.10%
+42.1
To assess the quality of the newly generated molecules from StitchNet, we measured the improvement and non-improvement in objective scores (GSK33, JNK3, QED, SA) between the stitched molecules and the existing molecules in the offline dataset. Table 20 presents the results, showing the percentage of improvement and non-improvement. Compared the existing molecules in the offline dataset, the newly generated molecules from StitchNet exhibited significant increases in challenging objectives such as GSK3,9 and JNK3, while showing slight decreases in easier-to- toster-t--optimize objectives like QED and SA. This suggests that StitchNet effectively provides diversity beyond the offline dataset and enhances performance in challenging objectives with only a minor reduction easier objectives. Consequently, the generative model can learn from this enriched set of high-quality molecules generated by StitchNet, leading to an overall improvement in performance.
Q. Reward Hacking Problem in Offline Multi-Objective Molecular Optimization
In this study, we tackle the offline multi-objective molecular optimization problem, which requires optimizing multiple molecular objectives simultaneously. Throughout this optimization process, we observed conflicts between certain molecular objectives. To investigate these conflicts further, we conducted a detailed analysis of each property score within a four- objective scenario (GSK33, JNK3, QED, and SA).
We found that models often prioritized easier objectives, such as QED and SA, over more challenging objectives like GSK333 and JNK3. As reported in previous study (Gao et al., 2022). QED is often considered too trivial, allowing most models to achieve high scores this objective with minimal effort. This suggests that increasing and optimizing the QED score is much simpler compared to tackling more challenging objectives. For instance, models like REINVENT, which receive rewards based on the average property score, may focus on easily attainable objectives to maximize the overall reward. Consequently, this creates the reward hacking problem. where the model overfit to easier objectives while neglecting the more challenging ones. This behavior highlights the inherent difficulty in multi-objective optimization, particularly when some objectives are easier to optimize than others.
45

---

Offline Model-based Optimization for Real-Worid Molecular Discovery
One possible approach to address this issue could be adjusting the weights assigned to each objective to balance their influence-placinnn more emphasis on the challenging objectives and less the easier ones. However, this approach relies on having prior domain knowledge about the difficulty of each objective, which is not always available. Moreover, in offline optimization settings, immediate feedback to refine weights is limited, making this approach impractical.
To overcome these challenges, we introduced priority sampling using a Dirichlet distribution within our MolStitch framework for Pareto optimization. This approach efficiently generates diverse weight configurations, ensuring a balanced exploration of all objectives. By using priority sampling within our framework, we promote the generation of a diverse set of stitched molecules that do not disproportionately favor easier objectives, thereby mitigating the risk of reward hacking.
Table 21. Property scores for each objective in a four-objective scenario (GSK38, JNK3, QED, SA).
QED
SA
JNK3
GSK33
w/o MolStitch
0.843
0.889
0.128
0.397
MolStitch (Ours)
0.709
0.802
0.485
0.688
To validate the effectiveness of our MolStitch framework, we compared the property scores for each objective in a four- objective scenario (GSK333, JNK3, QED, and SA) before and after applying our MolStitch framework that incorporates priority sampling. The results. presented in Table 21, clearly indicate that without MolStitch, the models suffer from the reward hacking problem, achieving disproportionately high scores easier objectives like QED and SA while exhibiting extremely low scores on more challenging objectives such as JNK3 and GSK33. In contrast, applying our MolStitch framework results in a more halanced optimization, with relatively improved and well-distributed scores across all objectives.
R. Chebyshev Scalarization
While our initial implementation of MolStitch employed linear scalarization due to its simplicity and foundational role in multi-objective optimization, we also explored Chebyshev scalarization to assess its potential advantages. Specifically, Chebyshev scalarization minimizes the maximum weighted deviation from reference point, which is typically defined as the ideal vector in objective space. Unlike linear scalarization, which aggregates all objectives into a single weighted sum, Chebyshev scalarization evaluates each objective independently and focuses on minimizing the worst worst-performing objective. This mechanism encourages more balanced solutions and enables the algorithm to explore non-convex and concave regions of the Pareto front, which are often inaccessible to linear scalarization techniques (Deb et al., 2016).
To empirically assess the influence of Chebyshev scalarization within our MolStitch, we conducted experiments across two-objective (GSK3,3+JK33 three-objective (GSK3 optimization tasks. As shown in Table 22, MolStitch with Chebyshev scalarization achieved comparable performance to the linear approach in the two-objective setting. This is consistent with our expectation that the Pareto front remains relatively simple and convex in wwwdiimensional spaces, limiting the potential benefits of more sophisticated scalarization techniques. In contrast, Chebyshev scalarization consistently outperformed linear scalarization in both three- and four-ohjective settings. This improvement likely due to the increased complexity and non-convexity of the Pareto front in higher dimensions, where Chebyshev's emphasis extreme deviations facilitates more effective balancing of trade-offs across objectives.
and four-objective (GSK3 (-SK33+-NK3+
Table 22, Additional experimental results on the molecular property optimization task using the Chebyshev scalarization technique.
Molecular objectives
GSK3,8+JNK3
GSK33-+IN3+OEED
GSK18+NN3+QED+SA
Method
HV(1)
R2(J)
HV(1)
R2(1)
HV(t)
R2(()
MolStitch (w/ (w/LLnean) Linear)
0.579
10.128
0.403
1.649910259
0.352_
2.953
MolStitch (w/ Chehyshev)
0.580-10.08
0.707-10.15
0.440-10.085
1.568
0.397
2.619-104533
We further analyzed the impact of Chebyshev scalarization on molecular diversity in the four-objective setting. Specifically, we measured the number of unique Bemis-Murcke (BM) scaffolds and carbon skeletons two widely used evaluation metrics of structural eiversity-amope the molecules located on the Pareto front. As demonstrated in Table 23, Chebyshev scalarization resulted in a notably more diverse set of molecules, as evidenced by the higher counts across both diversity metrics. This outcome further supports the assertion that Chebyshev scalarization promotes broader exploration of the solution space and facilitates the discovery of molecular candidates that might be overlooked by the linear approach.
46

---

Offline Model-based Optimization for Real-World Molecular Discovery
Table 23. Comparative analysis of diversity outcomes using Molstitch with linear and Chebyshev scalarization.
Molecular objectives
GSK31++NK3+0E4SSA
Diversity metrics
BM scaffold
Carbon skeletons
MolStitch (w/ Linear)
3453
1664
MolStitch (w/ Chebyshev)
3836
1976
In summary, these findings collectively demonstrate that MolStitch is compatible with both linear and Chebyshev scalariza- tion techniques. We think that linear scalarization remains a reasonable choice when the Pareto front is expected to be convex or when one objective is known to be dominant. However, Chebyshev scalarization is better suited for high-dimensional optimization tasks involving multiple, conflicting objectives. In such scenarios, Chebyshev can provide more robust trade-off among competing objectives and leads to the generation of more diverse and well-balanced molecular candidates.
S. Extended Analysis on Molecular Property and Docking Score Optimization Tasks
S.1. Additional Experimental Results on the Molecular Property Optimization Task
Given the relevance of the offline MOO benchmark (Xue et al., 2024) to study, we conducted additional experiments incorporating baseline methods introduced in that work, specifically the use of multiple models in conjunction with COMs and RoMA. The underlying idea of leveraging multiple models closely aligns with our ensemble proxy model setup in MolStitch. As shown in Table 24, the Multiple Models + COMs/RoMA configurations outperformed their single-model counterparts. This result is consistent with our findings throughout the paper, where ensemble-based proxy models generally demonstrated superior performance relative single-model approaches such as Grad.
Table 24, Additional experimental results on the molecular property optimization task, including baselines from the offline MOO paper.
Molecular objectives
GSK3
GSK33+JNK3+
CSK38+-NK3+QED-SA
Method
HV(t)
R2(()
HV(t)
R2(J)
HV(1)
R2(()
Single Model + COMs
0.479.10.063
0.877 10.1000000 109
0.205 100072
2.496.10.288
0, 100062
4.219110628
Single Model + RoMA
0.492=10.091
0.843
0.198:00.052
2.537410269
0. 0.169
4.207:+0.617
Multiple Models + COMs
0.489:10.089
0.814=0.117
0.21 1 +00005
2.449:10.407
0.190=+0.061
4.157:20.592
Multiple Models + RoMA
0.499:10.081
0.812::0.141
0.214-10.050
2.472:100227
0. 00.007
3.988:10.533
REINVENT-BO
0.472:10.107
0.0211
10.086
2.385:10.393
1.0105
3.974:10.895
MolStitch (Ours)
0.579
0.698=0.128
0.4033+0.065
1.649+40259
0.352++0.00
2.953++0571
In addition, motivated by the strong performance of Bayesian optimization (BO) reported in the offline MOO benchmark, we implemented comparable BO approach using REINVENT as the generative model, which we refer to as REINVENT-BO. As shown in the same table, REINVENT-BE achieved competitive results, indicating its viability baseline method for offline molecular optimization tasks. Nevertheless, our MolStitch framework consistently outperformed all these bascline methods, thereby reaffirming the robustness and efficacy of our approach.
S.2. Advancing MolStitch with State-of-the-Arr Techniques for the Molecular Property Optimization Task
To further enhance the capabilities of MolStitch, we explored the integration of both well-established and state-of-the-art techniques, including simulated annealing (Van Laarhoven et al., 1987) and the Pagment-RAG(L et al., 2024) methods. First, simulated annealing (SA) is a probabilistic optimization algorithm that improves exploration by allowing the acceptance of suboptimal solutions with certain probability in order to escape local optima. Inspired by this principle, we introduced an SA mechanism into MolStitch by allowing the occasional acceptance of losing molecules during training. As shown in Table 25, MolStitch with SA did not lead significant performance gains. We attribute this marginal benefit of additional exploration through SA to the effectiveness of our rank-based proxy model in reliably distinguishing between candidate molecules, well as the intrinsic diversity already promoted by the StitchNet architecture.
We also investigated incorporating ideas from fragment-RAN RAG (f-RAG), which is a recent state-of-the-an framework that introduces the use of hard cecpments-xxppiccn structural components used to construct new molecules- and soft fragments, which are injected as embeddings to implicitly guide the generation process. Since StitchNet naturally supports the principles
47

---

Offline Model-based Optimization for Real-World Molecular Discovery
Table 25. Additional experimental results the moleeular property optimization task, integrating well-estahlished and recent techniques.
Molecular objectives
GSK3Ø+JNK3
GSK38+INK3+QED
GSK33+JNKS+QED4SA
Method
HV(()
R2(4)
HV(()
R2(4)
HV(1)
R2(()
MolStitch
0.579=00070
0.698+0.128
0.403
1.649=00259
0.352=+0.080
2.953=0.571
MolStitch (w/ SA)
0.542
0.774:10.200
0.0044
1.669110.12
0.361 1:0040
2.704=0.276
MolStitch (w/ f-RAG)
0.578+00.09
0.694:1.111
1.0081
1.519=0.277
0.412:10.060
2.494=0.356
of hard fragments through explicit recombination of molecular substructures, readily accommodates the first component of f-RAG. To incorporate soft fragment guidance, extended StitchNet condition on fragment embeddings during the stitching process, enabling implicit control over molecule generation. As demonstrated in Table 25, MolStitch with f-RAG yielded improved performance, highlighting the benefits and effectiveness of soft fragment guidance.
S.3. Additional Experimental Results on the Docking Score Optimization Task using SMINA
In our main study, employed Quick (QVina) (Alhossary et al., 2015) for the docking score optimization task, which is a widely recognized molecular docking tool derived from AutoDock Vina. Our decision to use QVina was motivated by its widespread adoption and following established practices from prior studies (Lee et al., 2023; Guo & Schwaller, 2024b). While Q is widely used, other molecular docking tools derived from AutoDock Vina have been developed, among which SMINA (Cieplinski et al., 2023) is particularly notable. Specifically, SMINA offers extended capabilities, including support for customized scoring functions and the ability enforce specific ligand-receptor receptor interaction constraints.
Table 26. Additional experimental results docking score optimization tasks using SMINA the docking evaluation tool.
Target protein
parpl
jak2
braf
fa7
Shtlb
Method
HV(1)
HV(T)
HV(()
HV(f)
HV(f)
REINVENT
0.522:10.007
0.476++0004
0.503-1:0006
0.417:10.004
0.514:10003
BootGen
0.534-10.006
0..089
0.008:10.000
0.00070
0.530:100016
MolStitch (Ours)
0.550:10.007
0.539-10.060
0.530:10.012
0,450:10.004
0.544:1000M
Table 27, Additional experimental results on docking score optimization tasks using SMINA as docking evaluation tool.
Target protein
parpl
jak2
braf
fa7
5ht1b
Method
R2(()
R2(4)
R2(1)
R2(()
R2(()
REINVENT
1.408:40:021
1.589-40.58
1.487:00.05
1.804:40009
1..030
BootGen
1.362.10.044
1.502.10.044
1.444.10.066
1.772:10.032
1.376:10.080
MolStitch (Ours)
1.303
1.325.10.177
1.397.10.04
1.627.10.06
1.329.10.00
Given its relevance and growing adoption, we conducted additional docking score optimization experiments using SMINA
to further evaluate the robustness of our MolStitch framework. As shown in Tables 26 and 27, MolStitch consistently maintained superior performance terms of both HV and R2 evaluation metrics, even when assessed using SMINA. These findings further validate the generalizability and robustness of our MolStitch across distinct molecular docking tools.
48

---

Offline Model-based Optimization for Real-World Molecular Discovery
T. Future Work and Limitations
In this study, we focused on optimizing the properties small molecules and docking scores for five specific proteins. natural extension this work would to apply our framework to material discovery, particularly for optimizing inorganic molecules, thereby broadening its applicability beyond small molecules. Additionally, while we investigated both full-ofiline and semi-offline optimization settings, there remains considerable potential enhance the semi-offline optimization. One promising direction is the use of behavior policy improve exploration of chemical space when periodically incorporating new molecule data. This strategy would enable the inclusion of molecules that were not present in the initial offline dataset, leading to more effective integration of newly obtained data. Even in cases where the initial offline dataset contains lower-quality molecules, behavior policy could progressively improve the quality of the data over time. Moreover, our results suggest that employing multiple proxies yields valuable insights and substantial performance gains in specific cases. As such, future work will focus on further developing and optimizing multiple proxy methods fully realize their potential in molecular discovery.
U. Molecule Examples
In this section, we provide visual examples of molecules generated by the fine-tuned generative model, which aims to produce novel molecules that surpass the best-known molecules in the offline dataset. Specifically, we present representative molecules sampled from the Pareto front the four-objective optimization scenario (OBD+SA+INK3-GS333)) Each molecule illustrates distinct trade-off among these objectives, demonstrating the diverse range of solutions on the Pareto front. These examples emphasize the ability of our framework to explore diverse molecules that effectively balance multiple objectives.
49

---

Offline Model-based Optimization for Real-Worle Molecular Discovery
Jus 1000000
B [AAAS00MM04]]
DLAMAA DOSAAS]
[2 (434004000004
(89R000E 14039)
[131092075000
(0H001000000)
(334388825004)
DESOBAC 82005
02030)
IDELOREE DOODK
JUHE B4000AAX
IBRUNNCL 50000)
pannass a (OMM)
MO0DRBE444 90)
JUSB BULLIDE 200)
30 30000000000
DA5S080 FREWS
Figure Representative molecules sampled from Pareto in four-objective optimization scenario (QED4SA+INK++GSKS The numerical scores for each objective are displayed below the respective molecular structures. Each molecule reflects distinct trade-off among these objectives, highlighting diverse range solutions on the Pareto front
50


Reviews:
[{"review_id": "EdfoyyaWWw", "reviewer": "Reviewer_B6bd", "summary": "This paper introduces MolStitch, a framework for offline molecular optimization that generates novel molecules by \"stitching\" fragments from an existing offline dataset, eliminating the need for iterative oracle queries. Inspired by trajectory stitching in offline reinforcement learning, MolStitch uses StitchNet to combine desirable properties from parent molecules and a rank-based proxy to evaluate molecules through pairwise comparisons. To address multi-objective optimization, the framework employs priority sampling with a Dirichlet distribution to explore diverse trade-offs along the Pareto front. Experimental results demonstrate MolStitch's effectiveness in outperforming existing methods across various offline molecular optimization benchmarks.\n\n## update after rebuttal\n\nThe authors provided a strong rebuttal with new experiments and detailed clarifications. They showed that MolStitch benefits from Chebyshev scalarization in complex multi-objective settings, both in performance and molecular diversity. They also added comparisons to ICML 2024 baselines, including Multiple Models + COMs/RoMA and REINVENT-BO, where MolStitch remained superior. Finally, their justification of the design choices—molecular stitching, rank-based proxy, and priority sampling—was well-motivated by domain-specific insights. This is why I raised my score to accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your thoughtful feedback. Below, we respectfully provide point-by-point responses to each of your comments.\n# Q1: Have you explored the Chebyshev scalarization technique?\nIn our original study, we employed linear scalarization because it is one of the most fundamental scalarization techniques, aiming to establish the applicability of the MolStitch framework at a fundamental level. \n\nIn response to your suggestion, we conducted additional experiments using MolStitch with Chebyshev scalarization in place of linear scalarization. Due to character limitations, we have uploaded the full results tables at the following link [here](https://tinyurl.com/molstitch).\n\nAs shown in the table, **MolStitch w/ Chebyshev** achieved comparable performance to the linear approach in the two-objective setting. Notably, it outperformed the linear approach in the more challenging three-objective and four-objective settings.\n||JNK3+GSK3β|JNK3+GSK3β+QED|JNK3+GSK3β+QED+SA|\n|-|:-:|:-:|:-:|\n||HV|HV|HV|\n|w/ Linear|0.579|0.403|0.352|\n|w/ Chebyshev|0.580|0.440|0.397|\n\nWe believe this is because in the two-objective setting, the Pareto front remains relatively simple, limiting the observable benefits of Chebyshev. As the number of objectives increases, the Pareto front becomes more complex and non-convex, which allows the advantages of Chebyshev scalarization to become more pronounced. Overall, these results show that MolStitch works well with both linear and advanced Chebyshev scalarization techniques. \n# Q2: Include baselines from the ICML 2024 offline MOO paper (Multiple Models + COMs/RoMA).\nFollowing your suggestion, we conducted additional experiments with Multiple Models + COMs/RoMA from the offline MOO paper. As shown in the table, **Multiple Models + COMs/RoMA** demonstrated better performance than their single model counterparts. This finding aligns with results throughout our paper, where ensemble proxy generally outperformed single proxy methods such as Grad.\n\n||JNK3+GSK3β|JNK3+GSK3β+QED|JNK3+GSK3β+QED+SA|\n|-|:-:|:-:|:-:|\n||HV|HV|HV|\n|Single Model+COMs|0.479|0.205|0.171|\n|Single Model+RoMA|0.492|0.198|0.169|\n|Multiple Models+COMs|0.489|0.211|0.190|\n|Multiple Models+RoMA|0.499|0.214|0.188|\n|REINVENT-BO|0.472|0.232|0.205|\n|MolStitch|0.579|0.403|0.352|\n\nAdditionally, inspired by the strong Bayesian optimization (BO) performance in the offline MOO paper, we implemented a comparable BO method using REINVENT as the generative backbone. We refer to this as REINVENT-BO. As shown in the table, **REINVENT-BO** achieved competitive performance, highlighting its potential in this domain. However, our MolStitch framework consistently outperformed all these baselines, thereby reaffirming the superiority and efficacy of our approach.\n# Q3: Can you explain why the combination of trajectory stitching, rank-based proxy, and priority sampling is particularly effective for molecular discovery in a multi-objective setting?\n* Inspired by trajectory stitching, we propose a novel **molecular stitching** operation that recombines fragments from two parent molecules to create new stitched molecules. In molecule design, it is well established that core molecular substructures—often referred to as privileged scaffolds—play a critical role in determining key biological properties [1]. By recombining these scaffolds from diverse parents, molecular stitching creates molecules that inherit beneficial traits, enabling a broader exploration of chemical space and enhancing the discovery of novel, diverse, and high-quality candidates.\n\n* However, evaluating these newly stitched molecules presents a practical challenge: they often fall outside the distribution of the training data, making it difficult for conventional proxy models to reliably approximate their property scores. To address this, we introduce a **rank-based proxy**. Rather than regressing absolute property values, our proxy learns to predict which of two molecules is more likely to be superior with respect to the target properties. This classification-style formulation simplifies the learning task and enhances the robustness of the proxy model.\n\n* In a multi-objective setting, fixed weight configurations often struggle to balance competing objectives like potency and safety. Our **priority sampling** mechanism addresses this by generating diverse weight configurations that emphasize different objectives. This approach encourages the model to maintain a diverse population of candidate molecules that span a wide range of trade-offs. For example, in drug discovery, one scenario may prioritize potency over safety (e.g., cancer treatments), while another may require the opposite (e.g., pediatric applications). Priority sampling enables the exploration of these diverse trade-offs among multiple objectives, providing domain experts with a broader range of candidate molecules.\n\n[1] Welsch et al. “Privileged scaffolds for library design and drug discovery.” Curr Opin Chem Biol. (2010).", "claims_and_evidence": "Claim:  Linear scalarization with priority sampling enables effective exploration of trade-offs in multi-objective optimization.\n\nIssue: While priority sampling with a Dirichlet distribution is effective for generating diverse weight configurations, the reliance on linear scalarization assumes a convex Pareto front. This assumption may not hold in practice, as Pareto fronts in molecular optimization can often be non-convex. The paper does not provide evidence or discussion on how this limitation affects performance in such scenarios. Including additional experiments or theoretical analysis on non-convex Pareto fronts would strengthen this claim and provide a more comprehensive understanding of the framework's capabilities.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are well-aligned with the problem of offline molecular optimization. MolStitch introduces StitchNet to generate novel molecules by combining fragments from an offline dataset and employs a rank-based proxy for robust evaluation through pairwise comparisons. These methods effectively address the challenge of optimizing molecules without repeated oracle queries. The evaluation uses standard benchmarks (e.g., molecular property and docking score optimization) and metrics (e.g., hypervolume, R2.\nHowever, one limitation is the use of linear scalarization for multi-objective optimization, which assumes a convex Pareto front—an assumption that may not always hold in practice.", "theoretical_claims": "They look good.", "experimental_designs_or_analyses": "The experimental designs and analyses in the paper are sound and valid, with appropriate benchmark datasets, evaluation metrics, and comparisons to state-of-the-art methods. The ablation studies further strengthen the validity of the results.", "relation_to_broader_scientific_literature": "The paper addresses offline multi-objective optimization (MOO) in molecular discovery, building on prior work in offline single-objective optimization (SOO) and preference optimization (e.g., DPO, IPO). It introduces MolStitch, which combines trajectory stitching from offline RL and a rank-based proxy for robust evaluation. While the individual components are not entirely novel, their integration into offline MOO is timely, especially given the limited work in this area, as highlighted by the recent ICML 2024 \"Offline Multi-Objective Optimization\". That paper underscores the importance of offline MOO in real-world applications like molecule and protein design, proposing benchmarks to advance the field. \n\nMolStitch’s focus on avoiding costly oracle evaluations makes it a practical and valuable tool for practitioners, even if it does not introduce groundbreaking innovations.", "essential_references_not_discussed": "Yes, the paper misses a key reference: the ICML 2024 paper \"Offline Multi-Objective Optimization\" by Ke Xue et al. This work is highly relevant because it introduces a wide range of baselines for offline MOO, including methods like Multiple Models + COMs, Multiple Models + RoMA, and Multi-head approaches. These baselines are essential for comparing and evaluating new methods like MolStitch. By not citing this work, the paper misses an opportunity to position MolStitch within the current state-of-the-art and demonstrate its performance against established baselines. Including this reference would provide a stronger foundation for understanding and validating the contributions of MolStitch.", "other_strengths_and_weaknesses": "Strengths:\n\n1- Practical Relevance: The paper addresses a highly relevant real-world problem: offline molecular optimization without costly oracle evaluations. This is significant for applications like drug discovery, where wet-lab experiments are time-consuming and expensive.\n\n2- Clarity and Presentation: The paper is well-written and clearly explains the motivation, methodology, and experimental results. The use of ablation studies and visualizations helps readers understand the contributions and effectiveness of the proposed framework.\n\n3- Strong Empirical Results: The paper demonstrates strong performance on benchmark tasks, outperforming several state-of-the-art methods. This empirical validation strengthens the significance of the work.\n\n\nWeaknesses:\n\n1- Linear Scalarization Limitation: The reliance on linear scalarization assumes a convex Pareto front, which may not hold in many real-world scenarios. The paper does not explore alternative scalarization techniques or provide evidence of performance on non-convex Pareto fronts, limiting its generalizability.\n\n2- Lack of Comparison with Offline MOO Baselines: The paper does not compare MolStitch with the baselines proposed in the ICML 2024 paper on offline multi-objective optimization. Including these comparisons would provide a clearer picture of how MolStitch performs relative to other state-of-the-art methods in offline MOO.\n\n3- Limited Justification for Combination of Ideas: While the paper combines ideas from offline RL (trajectory stitching), preference optimization (rank-based proxy), and multi-objective optimization (priority sampling), it does not provide a detailed explanation of why this specific combination is the best fit for molecular applications in a multi-objective setting. A more thorough discussion of the rationale behind these choices would strengthen the paper.", "other_comments_or_suggestions": "No.", "questions_for_authors": "1- The framework uses linear scalarization, which assumes a convex Pareto front. How does MolStitch perform on problems with non-convex Pareto fronts, and have you explored alternative scalarization techniques (e.g., Chebyshev)?\n\n2- The ICML 2024 offline MOO paper proposes several baselines (e.g., Multiple Models + COMs/RoMA). Why were these not included in your experiments, and how does MolStitch compare to them in terms of hypervolume (HV) and R2 metrics?\n\n3- Can you explain why the combination of trajectory stitching, rank-based proxy, and priority sampling is particularly effective for molecular discovery in a multi-objective setting? Are there domain-specific insights that justify this design?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c9ci05ywQ4", "reviewer": "Reviewer_fpEX", "summary": "The paper introduces MolStitch, a framework for offline multi-objective molecular optimization (MOMO). Key contributions include StitchNet, which generates \"stitched molecules\" by combining fragments from an offline dataset; a rank-based proxy model for pairwise molecule evaluation; and preference optimization techniques (e.g., IPO) for fine-tuning the generative model. Priority sampling via a Dirichlet distribution is used to explore diverse trade-offs among objectives. Experiments on molecular property (MPO) and docking score optimization tasks demonstrate improvements in hypervolume (HV) and R2 metrics over baselines, with ablation studies validating the framework’s components. The method addresses challenges in offline settings where wet-lab evaluations are costly and slow.\n\n## update after rebuttal\n\nThe authors have addressed my comments including docking score optimization experiments using SMINA, and clarification on how StitchNet, the rank-based proxy contributes to final performance. Therefore, I will raise my score to accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your insightful feedback and the opportunity to clarify the key aspects of our study.\n# Q1: Why was SMINA not used for docking evaluation?\nIn our original study, we employed QuickVina (QVina) [1] for docking score evaluation, which is a widely recognized molecular docking tool derived from AutoDock Vina. Our decision to use QVina was motivated by its widespread adoption and following established practices from prior studies [2,3].\n\nWe appreciate your mention of SMINA [4]. Upon investigation, we found that SMINA is also derived from AutoDock Vina and provides additional capabilities, such as imposing constraints on ligand interactions. Due to its high relevance and in response to your valuable suggestion, we conducted additional docking score optimization experiments using SMINA. Due to character limitations, we have uploaded the full results tables at the following link [here](https://tinyurl.com/molstitch).\n\nAs demonstrated in the table, **MolStitch maintained its superior performance even when evaluated using SMINA**. These additional results further validate the robustness of our framework across different docking evaluation tools.\n\n||parp1|jak2|braf|fa7|5ht1b|\n|-|:-:|:-:|:-:|:-:|:-:|\n||HV|HV|HV|HV|HV|\n|REINVENT|0.522|0.476|0.503|0.417|0.514|\n|BootGen|0.534|0.498|0.518|0.425|0.530|\n|MolStitch|0.550|0.539|0.530|0.450|0.544|\n\nWe sincerely thank you for bringing SMINA to our attention, and we will include these additional results along with a description in our revised manuscript.\n# Q2: What is the novelty of StitchNet, and how does it differ from genetic algorithm-based crossover operators?\nConventional genetic algorithm-based crossover operators typically rely on hand-crafted, rule-based procedures to recombine fragments of parent molecules. Although these rules incorporate chemical intuition and domain expertise, they remain fixed throughout the search process and do not adapt based on the quality of generated outcomes. Consequently, rule-based crossover operators may explore chemical space less effectively, as they lack mechanisms to refine and favor recombination strategies that produce higher-quality molecules.\n\nIn contrast, StitchNet employs a neural network architecture trained via self-supervised learning to adaptively discover effective fragment recombination strategies. **Specifically, StitchNet leverages chemical feedback—using objective scores from an offline dataset—to guide the stitching process**, ensuring that resulting molecules are chemically valid and likely to exhibit desirable properties. This self-supervised learning mechanism allows StitchNet to refine its recombination strategies based on chemical feedback, enabling it to explore promising regions of chemical space.\n# Q3: Are the gains primarily from StitchNet, the rank-based proxy, or their combination?\nThank you for the insightful question. As noted, the combination of StitchNet and the rank-based proxy is indeed synergistic and central to the effectiveness of our framework. However, we agree that a clearer breakdown of their individual contributions is valuable.\n\nIn Appendix C.6 of our manuscript, we provided a comprehensive ablation study that isolates the effects of each component. From these results, we observed that the addition of the rank-based proxy alone yields greater performance improvements compared to the addition of StitchNet alone.\n\nTo clarify the ablation study experimental setup:\n* In the **rank-based proxy alone** setting, new molecules are sampled directly from the generative model (`model.sample()`) and evaluated using the rank-based proxy. The generative model is then updated through preference optimization based on this proxy feedback.\n* In the **StitchNet alone** setting, new stitched molecules are generated by StitchNet and evaluated using a score-based proxy that directly estimates their objective scores. The generative model is subsequently updated using these estimated scores as pseudo-rewards.\n\nWe believe the rank-based proxy alone performs more effectively than StitchNet alone because it provides robust and reliable feedback, enabling stable and meaningful updates to the generative model. In contrast, although StitchNet alone enhances molecular diversity by producing novel stitched molecules, the score-based proxy often struggles to evaluate them accurately, leading to less reliable feedback. **In essence, the full potential of our framework is achieved when these two components are combined, effectively leveraging their complementary strengths.**\n\n[1] Alhossary et al. “Fast, accurate, and reliable molecular docking with QuickVina 2.” Bioinformatics (2015).\n\n[2] Guo et al. “Saturn: Sample-efficient generative molecular design using memory manipulation.” arXiv (2024).\n\n[3] Lee et al. “Drug Discovery with Dynamic Goal-aware Fragments.” ICML (2024).\n\n[4] Cieplinski et al. “Generative models should at least be able to design molecules that dock well: A new benchmark.” JCIM (2023).", "claims_and_evidence": "The claims are supported by experiments across multiple tasks, ablation studies, and comparisons to diverse baselines. However, the evidence has limitations:  \n1. Benchmark coverage: it is not clear why recent docking score benchmark SMINAare not included.\n\n2. Component contributions: The improvements from StitchNet vs. other data augmentation methods (e.g., crossover operators) are shown, but the analysis lacks depth in explaining why neural-based stitching outperforms other genetic algorithm alternatives such as Saturn.  \n\n3. Rank-based proxy: The proxy’s superiority over score-based variants is demonstrated via accuracy metrics, but its impact on downstream optimization (beyond pairwise classification) is not thoroughly analyzed.", "methods_and_evaluation_criteria": "The methods are sensible for offline MOMO, leveraging stitching and preference optimization. However:  \n- Benchmark choice: The omission of SMINA (a docking benchmark not discussed in related work) is unexplained, weakening the docking score evaluation’s credibility.  \n- Metrics: HV and R2 are appropriate for multi-objective tasks, but the paper does not clarify why these metrics were prioritized over others (e.g., success rate, diversity scores).", "theoretical_claims": "The paper does not present theoretical claims, focusing instead on empirical validation. No theoretical issues are noted.", "experimental_designs_or_analyses": "Semi-offline experiments: Results are deferred to the appendix, limiting insight into this practically relevant setting.  \n- Backbone model: While REINVENT is justified as a backbone, newer architectures (e.g., GFlowNets, Mamba) are only briefly discussed in appendices.", "relation_to_broader_scientific_literature": "The work connects to offline MBO and preference optimization literature but could better contextualize:  \n- Trajectory stitching: The analogy to RL trajectory stitching is under-explored; differences in molecular vs. RL state spaces are not discussed.  \n- Preference optimization: While DPO/IPO are applied, their adaptation to molecular design (vs. language models) lacks critical analysis and insights.", "essential_references_not_discussed": "- SMINA docking tool: Critical for docking score benchmarks, as highlighted in Ciepliński et al.’s work.  \n- Recent preference-based molecular design**: Works like below apply preference optimization to molecular optimization but are not cited.  \n\n1. Extracting medicinal chemistry intuition via preference machine learning\nOh-Hyeon Choung, Riccardo Vianello, Marwin Segler, Nikolaus Stiefl & José Jiménez-Luna \nNature Communications volume 14, Article number: 6651 (2023) \n\n2. Preference Optimization for Molecular Language Models", "other_strengths_and_weaknesses": "Strengths: The integration of stitching, rank-based evaluation, and preference optimization is novel. The semi-offline experiments and diversity analysis (e.g., Bemis-Murcko scaffolds) are practical contributions.  \n\nWeaknesses: The components (stitching, ranking, priority sampling) are largely adaptations of existing ideas, limiting conceptual novelty. The writing is dense, with insufficient intuition for non-experts.", "other_comments_or_suggestions": "The paper presents a technically sound framework with thorough empirical validation, but the incremental nature of contributions and benchmark omissions temper enthusiasm. Addressing the questions above could strengthen the case for acceptance.", "questions_for_authors": "1. Benchmark justification: Why was SMINA not used for docking evaluation, despite its relevance in the cited benchmark?  \n\n2. StitchNet vs. crossover: How does StitchNet’s neural approach fundamentally differ from genetic algorithm-based crossover operators in exploring chemical space?  \n\n3. Component dominance: Are the gains primarily from StitchNet, the rank-based proxy, or their combination? Ablation suggests synergy, but a breakdown would clarify.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xPOHi7dgmB", "reviewer": "Reviewer_tueg", "summary": "This paper introduces the Molecular Stitching (MolStitch) framework, designed to address the molecular discovery problem in an offline setting, where an offline dataset is employed without requiring iterative queries to the oracle function. Particularly, MolStitch operates by leveraging existing molecules from the offline dataset to generate novel stitched molecules that combine desirable properties using the StitchNet model. A rank-based proxy model is then employed to compare molecules in the stitched set, determining which is preferable in each pair. This information is used to fine-tune the generative model through Identity Preference Optimization (IPO). The effectiveness of the MolStitch framework is demonstrated through two key offline MOMO experiments, showcasing its potential in molecular optimization.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are truly grateful for your thoughtful feedback. In the following, we carefully respond to each of your comments.\n# Q1: The authors should include recent works\nWe sincerely thank the reviewer for highlighting these important references [1–5], which are indeed highly relevant from the perspective of offline optimization. We fully agree with the significance of these works and will ensure they are properly cited and discussed in the revised manuscript.\n\nRegarding the method presented in [1], RaM, we appreciate the reviewer’s suggestion to emphasize its connection to our work. While we had included RaM as a competing baseline and compared its performance with our MolStitch framework in Tables 1 and 2 of the main manuscript (with additional details in Appendix N), we acknowledge that we had not discussed it explicitly in the related work section. \n\nWe are grateful for the reviewer’s suggestion and will make sure to address this omission in the revision to provide a more complete overview of related work. Additionally, since an extra page is permitted for the revised manuscript, we plan to move the main related work section into the main text, while retaining the extended discussion in Appendix.\n\n# Q2: Where is the pre-training process of the generative model described in this paper?\nIn this work, we used REINVENT as the main backbone generative model, both for our MolStitch and for all baseline offline optimization methods to ensure fair comparison. REINVENT is an RL-based generative model that produces molecules in an auto-regressive manner.\n\nAs part of its standard training pipeline, REINVENT first undergoes a pre-training process, where it learns to generate chemically valid molecules by capturing the underlying chemical grammar. This pre-training process is critical to ensure that the model can produce syntactically valid molecular structures before any optimization takes place. \n\nFollowing pre-training, REINVENT is further optimized via reinforcement learning, where it receives feedback in the form of reward signals based on the desired molecular objectives. This two-stage pipeline—pre-training followed by fine-tuning—has proven highly effective in the molecular domain, enabling REINVENT to achieve robust performance across a range of molecular optimization tasks.\nWe acknowledge that our original manuscript did not provide sufficient detail about the pre-training process for the generative model, and we will include a more comprehensive explanation in the revised manuscript.\n\n# Q3: Even minor structural modifications to a molecule can lead to significant variations in certain properties.\nIn our study, StitchNet is trained through self-supervised learning by decomposing a single molecule into two fragments and then recombining them. Because the fragments originate from the same parent molecule, the stitched molecule is likely to preserve essential substructures (core scaffolds) that strongly influence molecular properties. Moreover, our approach is supported by the Similar Property Principle (SPP) [6], a foundational concept in drug discovery and QSAR research, which states that structurally similar molecules often exhibit similar properties. To further ensure sufficient structural similarity, we employed Tanimoto similarity metric between original molecules ($m_{orig}$) and their stitched counterparts ($\\bar{m}_{stit}$). Prior research [7] has indicated that molecules with a Tanimoto similarity greater than 0.887 commonly demonstrate similar biological activities.\n\nHowever, we acknowledge exceptions to this principle, particularly in cases involving stereochemistry or activity cliffs, where minor structural changes can lead to major shifts in molecular properties. To address these limitations, we plan to incorporate advanced fingerprints—such as 3D-aware or chirality-aware descriptors—that capture more detailed structural and spatial information. We will include these considerations in the limitations and future work section of our revised manuscript.\n\n# Q4: Authors could consider rearranging the order of stages.\nThank you for the helpful suggestion. We agree that rearranging the order of stages would improve the logical flow, and we will revise it accordingly.\n\n[1] Tan et al. “Offline Model-Based Optimization by Learning to Rank.” ICLR (2025).\n\n[2] Dao et al. “Boosting offline optimizers with surrogate sensitivity.” ICML (2024).\n\n[3] Nguyen et al. “Expt: Synthetic pretraining for few-shot experimental design.” NeurIPS (2023).\n\n[4] Hoang et al. “Learning surrogates for offline black-box optimization via gradient matching.” ICML (2024).\n\n[5] Chemingui et al. “Offline model-based optimization via policy-guided gradient search.” AAAI (2024).\n\n[6] O’Boyle, Sayle. “Comparing structural fingerprints using a literature-based similarity benchmark.” J Cheminform (2016).\n\n[7] Cheng et al. “Investigating the correlations among chemical structures, bioactivity profiles and molecular targets.” Bioinformatics (2010).", "claims_and_evidence": "The claims in this paper are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria are well-suited to the problem.", "theoretical_claims": "This paper does not include any theorems.", "experimental_designs_or_analyses": "I have verified the soundness and validity of the experimental designs and analyses.", "relation_to_broader_scientific_literature": "This paper addresses the offline multi-objective molecular optimization (MOMO) problem, which has promising applications in drug discovery and molecular design.", "essential_references_not_discussed": "No essential related works are absent that are crucial for understanding the key contributions of this paper.", "other_strengths_and_weaknesses": "Strengths:\n\n- Pre-training on the ZINC dataset helps StitchNet internalize chemical grammar, allowing it to generate chemically valid stitched molecules.\n- Introduces a rank-based proxy for molecule evaluation instead of traditional value-based methods, followed by preference optimization to fine-tune the generative model.\n- The writing is clear and well-structured.\n- Empirical results support the effectiveness of the proposed method.\n\nWeaknesses:\n\n- The paper lacks a related work section in the main text.\n- While the mean performance over 10 different seeds is strong, the variance is high.", "other_comments_or_suggestions": "It is a minor point, but the authors could consider rearranging the order of stages. For example, Stage 1 could include the pretraining of the generative model and StitchNet, followed by the two proposed stages.", "questions_for_authors": "1. The authors should consider including and briefly describing recent works [1] [2] [3] [4] [5] on offline optimization in the Related Works section, particularly [1], as it shares a similar idea of training a rank-based proxy model.\n\n2. Where is the pre-training process of the generative model described in this paper? It would be helpful to clarify this aspect.\n\n3. Using the objective scores of $m_{orig}$ as chemical feedback to approximate the objective scores of $\\bar{m}_{stit}$ in Eq.(9) seems questionable. Even minor structural modifications to a molecule can lead to significant variations in certain properties. The authors should provide justification or additional validation for this approximation.\n\n[1] Tan, Rong-Xi, Ke Xue, Shen-Huan Lyu, Haopu Shang, Yao Wang, Yaoyuan Wang, Sheng Fu, and Chao Qian. \"Offline Model-Based Optimization by Learning to Rank.\" arXiv preprint arXiv:2410.11502 (2024).\n\n[2] Dao, Manh Cuong, Phi Le Nguyen, Thao Nguyen Truong, and Trong Nghia Hoang. \"Boosting offline optimizers with surrogate sensitivity.\" arXiv preprint arXiv:2503.04181 (2025).\n\n[3] Nguyen, Tung, Sudhanshu Agrawal, and Aditya Grover. \"Expt: Synthetic pretraining for few-shot experimental design.\" Advances in Neural Information Processing Systems 36 (2023): 45856-45869.\n\n[4] Hoang, Minh, Azza Fadhel, Aryan Deshwal, Janardhan Rao Doppa, and Trong Nghia Hoang. \"Learning surrogates for offline black-box optimization via gradient matching.\" arXiv preprint arXiv:2503.01883 (2025).\n\n[5] Chemingui, Yassine, Aryan Deshwal, Trong Nghia Hoang, and Janardhan Rao Doppa. \"Offline model-based optimization via policy-guided gradient search.\" In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 10, pp. 11230-11239. 2024.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iMJZPCaN3u", "reviewer": "Reviewer_RQBY", "summary": "The authors propose MolStitch as a generative method to generate molecular designs in an offline, multi-objective setting. Their method generates novel 'stitched molecules' that combine the desirable properties of original molecules sampled from the offline dataset. The authors evaluate their method on a number of offline molecular design tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your helpful feedback and the opportunity to enhance our manuscript.\n# Q1: Additional references would strengthen the experimental results.\nFirst, regarding **Simulated Annealing (SA)**, it is a probabilistic optimization algorithm that enhances exploration by occasionally accepting worse solutions with a certain probability. Inspired by this, we incorporated an SA mechanism into our MolStitch by occasionally accepting a losing molecule with a certain probability. Due to character limitations, **we have uploaded the full result tables at the following link** [here](https://tinyurl.com/molstitch). As shown in Table 5, MolStitch w/ SA did not yield noticeable performance improvements. We hypothesize this is because our rank-based proxy already performs well in determining winning and losing molecules, and the additional exploration facilitated by SA is somewhat redundant given StitchNet's diversity.\n\nSecond, we found **Fragment-RAG (f-RAG)** to be interesting. Although the official codebase was not available, we attempted to incorporate its core ideas into our framework. Specifically, f-RAG introduces the use of hard fragments—explicit structural components used to construct new molecules—and soft fragments, which are injected as embeddings to implicitly guide generation. StitchNet naturally supports hard fragments through explicit recombination of molecular substructures. To incorporate soft fragment guidance, we extended StitchNet to condition on embeddings of soft fragments during the stitching process. MolStitch w/ f-RAG achieved improved performance, highlighting the benefits of soft fragment guidance.\n\nThird and fourth, while **DyNA-PPO and GFNSeqEditor** are primarily designed for biological sequence design, they offer valuable insights from a model-based optimization perspective. We will include these important references in the related work section.\n# Q2: Include an ablation study with Chebyshev scalarization.\nThank you for this valuable suggestion. A similar point was also raised by another reviewer, which prompted us to conduct additional experiments. For a detailed discussion of the results, we respectfully refer you to our response to `Reviewer B6bd, Q1`.\n# Q3: In practice, most molecules are evaluated on only a subset of objectives.\nThank you for this insightful question. Several strategies have been proposed in the literature to address this challenge.\n* One direct approach is **imputation methods**, which estimate missing values from available data [1] or via pseudo-labeling in a semi-supervised manner [2].\n* Another strategy is **multi-task learning** [3], where models are trained to jointly tackle multiple objectives while allowing for missing labels. By sharing knowledge across related objectives, these models can leverage observed objectives to inform the learning of others. \n\nExtending our MolStitch to handle missing objective values would be an intriguing direction for future work. We will include these considerations in the limitations and future work section.\n# Q4: Additional details regarding the rule-based crossover operator.\nWe apologize for any confusion. The rule-based crossover operator ensures that child molecules generated from two parent molecules are chemically valid by following predefined chemical rules and constraints (e.g., SMARTS templates). Consequently, the goal of the unsupervised pre-training stage is to train StitchNet to imitate this rule-based crossover operator using a maximum likelihood estimation (MLE) objective, similar to teacher forcing.\n\nIn practice, we first generate numerous (parent1, parent2, child) triplets using the rule-based crossover operator. These serve as training examples for StitchNet, which is designed to generate child stitched molecules from given pairs of parent molecules. Although StitchNet may initially produce invalid stitched molecules, it gradually learns to imitate the rule-based crossover through MLE. Therefore, as pre-training progresses, StitchNet becomes increasingly proficient at generating chemically valid stitched molecules.\n# Q5: Assumptions for approximating objective scores of 𝑚̄ₛₜᵢₜ.\nThank you for this important question. A similar concern was also raised by another reviewer, and we respectfully refer you to our response to `Reviewer tueg, Q3`. Briefly, while we acknowledge the limitations, we tried to mitigate them by enforcing a similarity threshold to ensure stitched molecules retain sufficient structural overlap.\n# Q6: Is the loss function in Eq. (9) a summation or an expectation?\nThanks for the clarification. Eq.(9) is written as a sum over samples from the finite offline dataset.\n\n[1] Lobato et al. “Multi-objective genetic algorithm for missing data imputation.” Pattern Recognit Lett (2015).\n\n[2] Huang et al. “Offline data-driven evolutionary optimization based on tri-training.” Swarm Evol Comput (2021).\n\n[3] Liu et al. “Structured multi-task learning for molecular property prediction.” AISTATS (2022).", "claims_and_evidence": "The claims made by the authors are clear and supported by convincing evidence.", "methods_and_evaluation_criteria": "The molecular design tasks that the authors use to evaluate their proposed method are standard and representative of real-world molecular design tasks. I think this proposed methods and chosen evaluation benchmarks are well-motivated to study the problem proposed by the authors.", "theoretical_claims": "There are no theoretical claims to check the correctness of.", "experimental_designs_or_analyses": "The experimental design and analysis used by the authors are sound and use standard metrics from the offline optimization and multi-objective optimization literature (e.g., diversity of designs, Pareto fronts, hyper volume indicator, etc.", "relation_to_broader_scientific_literature": "In general, I think MolStitch is a meaningful contribution to the offline optimization for molecular discovery literature. The idea of using IPO for preference fine-tuning builds off of [IPO](https://arxiv.org/abs/2310.12036) and [DPO](https://arxiv.org/abs/2305.18290) from the language model literature, and the idea of trajectory annealing/stitching has been explore in prior work (e.g., [DiffStitch](https://arxiv.org/abs/2402.02439), [SSD](https://arxiv.org/abs/2402.07226), [GFNSeqEditor](https://openreview.net/forum?id=g0G8DQSBcj), [Fragment-RAG](https://arxiv.org/abs/2411.12078), Simulated Annealing). The authors also compare their method against relevant baselines in their experimental work.", "essential_references_not_discussed": "1. In general, I think most of the essential references have been discussed or included as relevant baselines. Additional references that I think would strengthen the experimental results include:\n\n  - [Simulated Annealing](https://en.wikipedia.org/wiki/Simulated_annealing)\n  - [Fragment-RAG](https://arxiv.org/abs/2411.12078) from Lee et al. Proc NeurIPS (2024).\n  - [DyNA-PPO](https://openreview.net/forum?id=HklxbgBKvr) from Angermueller et al. Proc ICLR (2020).\n  - [GFNSeqEditor](https://openreview.net/forum?id=g0G8DQSBcj) from Ghari et al. Proc NeurIPS (2024).\n  \nThat being said, I'm well aware that there are many offline MBO algorithms now proposed in the literature, and I think the authors have already demonstrated experimentally that their method works across a wide variety of different tasks. I would more strongly encourage Simulated Annealing and Fragment-RAG to be included as baselines given their similarity with the authors' proposed method. I feel less strongly that DyNA-PPO and GFNSeqEditor would need be included as baselines - appropriate discussion in the Related Work (if not already included) is likely more than sufficient.", "other_strengths_and_weaknesses": "In general, I think this is a well-motivated, well-executed, and well-written submission and lean towards recommending acceptance of this work. There are some additional experiments and associated discussion I recommend that would potentially help strengthen the paper that I detail in my earlier comments, but overall, I think this submission is sound.\n\n### Strengths\n\n2. The idea to use a DPO-/IPO- like framework for \"synthetic\" molecule priority sampling is interesting, original, and significant to the best of my knowledge.\n3. I appreciate the inclusion of batch hybrid learning results using the MolStitch method in the Appendix - this is a very real-world problem formulation and I the strong results of the authors' method in this setting strengthens the contributions of this paper.\n\n### Weaknesses\n\n4. The authors propose a method of objective scalarization via sampling from the Dirichlet distribution. A number of other method exist for scalarization - notably [Chebyshev scalarization](https://arxiv.org/abs/1904.05760) and even uniform sampling - that would be worth including as an ablation study.", "other_comments_or_suggestions": "None", "questions_for_authors": "5. In line 147, the authors assume that the offline dataset $\\mathcal{D}$ contains all of the evaluated objective scores for each of the $k$ objectives. In practice, I would imagine that the majority of molecules would only be experimentally evaluated using only a subset of the $k$ objectives in building the offline dataset. How would the method proposed by the authors (or how have others in prior work) adapt to this setting?\n6. Could the authors provide some additional details regarding the rule-based crossover operator using in the unsupervised pre-training stage? I am having a hard time understanding how this process should encourage StitchNet to internalize chemical grammar. More explicitly, my understanding is that molecules can have very similar token representations but represent very different molecules, and similar molecules in token space may have very different validity scores.\n7. In Section 3.2, the authors mention that they use the oracle score of $m\\_{orig}$ as an approximation for $\\bar{m}\\_{stit}$ because the stitched molecule shares the same molecular fragments as the original molecule. However, I would imagine that there might be functions that depend on the properties of the global molecule, or how the fragments are positioned with respect to one another in the molecule - such properties would be lost through the stitching process and make the approximation that $\\mathcal{R}$ is similar for the stitched and original molecules invalid. Is this the case? This is more so a minor clarification question on my part - I understand that the authors have cited prior work to support their argument (lines 215-217, right column), but am not as familiar with these prior literature.\n8. Is the loss function in Equation (9) indeed a summation over $m\\_{orig}\\in\\mathcal{D}$, or an expectation value?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: **Summary**: The paper considers the problem of offline optimization applied to the specific setting of molecular optimization where the search space is the space of molecules. The key idea of the paper is to develop a model that combines fragments from pairs of existing molecules in the offline dataset referred as “molecular stitching”. This acts like a data augmentation strategy as well. Additionally, the paper uses a learning to rank formulation for the surrogate/reward/proxy model. In order to extend the approach to the setting of multi-objective optimization, the paper considers sampling with a dirichlet distribution to explore diverse scalarizations of the objectives.

All the reviewers liked the paper and recognized the practical relevance of the problem setup and the proposed molecular stitching concept and the integration of rank-based preference learning. The authors’ response was convincing for the reviewers during the rebuttal. Overall, I am a bit skeptical of methods that have many moving parts without strong justification/unification. Such methods' performance tend to be brittle and break with small modifications in the problem setting. Having said that, **I am happy to recommend acceptance** based on unanimously supportive comments by the reviewers. I request the authors to update the final version based on discussion in the rebuttal period.
Decision: Accept (poster)