Molecule Design by Latent Prompt Transformer
Authors: Deqian Kong
Yuhao Huang
Jianwen Xie
Edouardo Honig
Ming Xu
Shuanghong Xue
Pei Lin
Sanping Zhou
Sheng Zhong
Nanning Zheng
Ying Nian Wu
Venue: NeurIPS 2024 spotlight (2024)
Keywords: latent space generative modeling; Latent Prompt Transformer; molecule design
Primary area: generative_models
TL;DR: We propose the Latent Prompt Transformer, a novel latent space generative model with both offline and online learning algorithms for molecule design.

Abstract: This work explores the challenging problem of molecule design by framing it as a conditional generative modeling task, where target biological properties or desired chemical constraints serve as conditioning variables.
We propose the Latent Prompt Transformer (LPT), a novel generative model comprising three components: (1) a latent vector with a learnable prior distribution modeled by a neural transformation of Gaussian white noise; (2) a molecule generation model based on a causal Transformer, which uses the latent vector as a prompt; and (3) a property prediction model that predicts a molecule's target properties and/or constraint values using the latent prompt. LPT can be learned by maximum likelihood estimation on molecule-property pairs. During property optimization, the latent prompt is inferred from target properties and constraints through posterior sampling and then used to guide the autoregressive molecule generation.
After initial training on existing molecules and their properties, we adopt an online learning algorithm to progressively shift the model distribution towards regions that support desired target properties. Experiments demonstrate that LPT not only effectively discovers useful molecules across single-objective, multi-objective, and structure-constrained optimization tasks, but also exhibits strong sample efficiency.

Molecule Design by Latent Prompt Transformer
Deqian Kong Yuhao Huang Jianwen Xie Edouardo Honig' Ming Xu², Shuanghong Xue*, Pei Lin Sanping Zhou', Sheng Zhong Nanning Zheng", Ying Nian Wu
DDepartment of Statistics and Data Science, UCLA
Institute of Artificial Intelligence and Robotics, Xi' an Jiaotong University BioMap Research "Akool Research
3 Institute of Engineering Medicine, UCSD
Shu Chien-Gene Lay Department of Bioengineering. UCSD
Abstract
This work explores the challenging problem of molecule design by framing it conditional generative modeling task, where target biological properties desired chemical constraints serve as conditioning variables. We propose the Latent Prompt Transformer (LPT), novel generative model comprising three components: (1) a latent vector with Iearnable prior distribution modeled by a neural transformation of Gaussian white noise; (2) molecule generation model based on a causal Transformer, which uses the latent vector as a prompt; and (3) property prediction model that predicts molecule's 's target properties and/or constraint values using the latent prompt. LPT can be learned by maximum likelihood estimation molecule- property pairs. During property optimization, the latent prompt is inferred from target properties and constraints through posterior sampling and then used guide the autoregressive molecule generation. After initial training on existing molecules and their properties, we adopt an online learning algorithm progressively shift the model distribution towards regions that support desired target properties. Experiments demonstrate that LPT not only effectively discovers useful molecules across single-objective, multi-objective, and structure-constrained optimization tasks, but also exhibits strong sample efficiency.
pooleule-property
1 Introduction
Molecule design plays pivotal role in drug discovery, focusing on identifying creating molecules with desired pharmacological or chemical properties. However, the vast and complex space of potential drug-like molecules presents significant challenges for systematically designing efficient machine learning models to navigate this space. Latent space optimization (LSO) (Maus et al., 2022; Tripp et al., 2020; Gómez-Bombarede et al., 2018), two-stage procedure, has emerged promising approach to address this challenge. The first stage involves training deep generative model, typically deep variational auto-encoder (VAE) (Kingma and Welling, 2014), to map oww-dimensional continuous vectors to the data manifold in input space, creating a simplified and continuous analog of the original optimization problem. In the second stage, the objective function is optimized over this learned latent space using surrogate model. While LSOO has demonstrated potential in tackling high-dimensional and structured input spaces, existing ISSO-based methods separate the training of the generative model from the property-conditione optimization. This decoupling can lead suboptimal performance, the learned latent space may be idcally suited to the specific
"Equal Contribution. Project page:
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

optimization task. Furthermore, LSO depends on an additional inference model during training, increasing the number of model parameters and adding complexity the model design process.
To address the above limitations, we propose a novel generative model, the Latent Prompt Transformer (LPT), which unifies molecule generation and optimization within single framework. The LPT can be directly trained on observed molecule-property pairs via maximum likelihood estimation (MLE) in an end-to-end fashion. We take advantage of the cemeediately-vvilabbee Markov chain Monte Carlo (MCMC) inference engine derived from the posterior distribution, eliminating the need for an auxiliary network for variational inference. The LPT consists of three components: (1) a learnable prior model of the latent vector based on neural transformation of Gaussian noise, (2) a molecule generation model that generates molecule sequences using a causal transformer with the latent vector serving as the prompt, and (3) property predictor model that estimates the target property values given the latent vector. The latent prompts serve as shared representations for both molecules and properties. By incorporating the MLE of the LPT, we employ an online learning algorithm of the LPT for property optimization, formulated as conditional generative modeling task. This algorithm progressively shifts the model distribution towards regions associated with desired properties.
To enhance the practical application of designed molecules and enable comparative analysis of our generated designs versus those made by human experts, we introduce new task, which is the conditional generation of molecules that bind to the NAD binding site of Phosphoglycerate dehydrogenase (PHGDH). In addition to single-objective optimization for this specific protein, we also introduce structure-comd conssrriied optimization to analyze the design pathways of learning- based models in comparison with human experts. This additional experiment aims to provide valuable insights for improving learning- compatibility with docking software, which is essential for facilitating precise score computation. Compared to other widely used evaluation metrics, the use of PHGDH offers greater practical significance and enables more accurate calculations in real-world drug discovery scenarios.
model design. We manually process the data to ensure
Our work makes the following key contributions: (I) We propose the LPT, a novel generative frame- work for jointly modeling molecule sequences and their target properties. This framework leverages a learnable informative prior distribution on latent space, conceptualized as an intrinsic design representation. (2) We propose an MCMC-based MLE to train the LPT without needing an auxiliary network for variational inference. (3) We develop a novel online learning algorithm for L.TT, allowing the model to gradually extrapolate to feasible regions associated with desired properties, improving computational and sample efficiency.
(4) We present a new task for the conditional generation of molecules that bind to the NAD binding site of PHGDH, and introduce structure-constraiiee optimization to compare the design pathways of learning-batt (5) Our model achieves sfate-of-the-art performance across a variety of molecule-based optimization tasks, including single-objective design, multi-objective design, and biological sequence design.
models with those human experts.
2 Background
Let (z) (1) (T) E X' be sequence representation of molecules such as SMILES (Weininger, 1988) or SELFIES (Krenn et al., 2020; Cheng et al., 2023), where 2x()) E V is the t-th element in the vocabulary V, and X' is the space of molecules. In molecule design, the objective is generate targeted molecules that optimize several properties of interest, rep- resented by
{u d((x) E |J for i= I,..., m, while satisfying specific constraints, c= (cp of E [2} for j = I,... n. Here, (f(x) and (x)) denote oracle functions de- termining property values and constraint satisfaction, respectively, and these values can be obtained either by querying existing software or by conducting wet lab experiments.
The multi-objective multi-constraint optimization (MMO) problem can be formulated as follows:
F(x) (z)( (z),, ... of (x)) s.t. 05(x)= 11, j 1=,,...,
max zEX
where (z(z)= = denotes constraint satisfaction and (x) = 0 denotes constraint violation. From probabilistic modeling perspective, the MMO problem can be viewed as conditional sampling problem, x= ~ p(zly = y", c = 1), where y" represents the desired values of the target properties, and e 1 indicates that all constraints are satisfied.
In generative molecule design, we assume access to a dataset of molecule sequences and their associated properties for offline pretraining of the generative model. During the design phase,
2

---

the model queries existing software (oracle functions) to obtain single or multiple metrics for the generated molecules, with these oracle functions providing ground-truth property values be optimized. It is important to acknowledge that developing proper software for accurate evaluation of molecule properties is of equal or even greater importance than designing molecules based on those software. However, it is beyond the scope of this paper to address the development of such software. Here, we treat the property values obtained from oracle functions as ground-truths for optimization.
3 Latent Prompt Transformer
3.1 Model
Suppose x = (,.())...,,'),,,, (."),,,,),....... ,..,, ,((1) is molecule sequence, (e.g. a string-based representation like SMILES (Weininger, 1988) or SELFIES (Krenn et al., 2020; Cheng al., 2023)), where x(() E V the i-th element of sequence in the vocabulary V. The latent vector is z E K and y E R is the target property value, y {0, 1} indicates constraint satisfaction. The term property refers to both real-valued properties and binary-valued constraints in the following sections. The joint distribution of molecule and its property is defined pæz, v)-
(1))
Cross-attention
(z0)
(o(2)
Causal Transformer
Pa(z|z)
(y(D)=)
Figure I: Left: Overview of Latent Prompt Transformer (LPT). The latent vector = IRd is a neural transformation of 2o, i.e., 2 = U. (zu), where ~ (N00, Ia). Given 2, x and are independent. p=(x]>> the molecule generation model and py (y]2z (z) predicts the property value or constraint based on z. Right: Illustration of molecule generation model pa (z] The latent vector is used prompt in the (=(x]=) via cross-altention.
The latent variable z, conceptualized as a design representation, decouples the molecule generation from property prediction. Specifically, given z, we assume a and are conditionally independent, making the information bottleneck. With this assumption, our Latent Prompt Transformer (LPT) is defined as,
po (1 2) (=()))p(=12)))
(2)
where 0 (a, B, 7). (z) a prior model with parameters a. Here, z serves as the latent prompt for the generation model pa (z|²) parameterized by a causal Transformer with parameters 3. p- (vl the predictor model with parameters 7. As shown in Fig. 1, LPT defines the generation process as, z ~ Pa (2), [x]2] ~ p=(2)2), [y] ~ (x(x)).
(3)
For the prior model, Pa is formulated as a learnalle neural transformation from an uninformative distribution, such an isotropic Gaussian, 2 U, (zn), where 20 ~ (0, La). U, (-) is parameter- ized by an expressive neural network such as Unet (Ronneberger et al., 2015) with parameters a. In this way, (2) can be viewed
implicit generator model.
The molecule generation model (=(=]²) is a conditional autoregressive model, ps(2[2) = Th= (=(x((t) .... _(t-1),,)) 2), which is realized by causal Transformer with parameters B. The latent vector z, serving as the prompt, controls every step of the autoregressive molecule generation. We incorporate these latent prompts into the (z(=)) through cross-attention, as shown in Fig. 1.
The real-valued property predictor is a non-linear regression model p. (y(z) = (x,, o²), where (2) is a small multi-layer perceptron (MLP) predicting based the latent prompt 2. The variance x² is treated as hyper-parameter that balances the exploitation-exploraiiio trade-off. For binary-valued constraints, Pt, (v]z) = (,) 8, 12) For multi-objective tasks, we can use heuristic-based combinations of multiple objectives to form special single-objective function.
3

---

3.2 Maximum Likelihood Learning of LPT
Suppose we observe training examples from the dataset with molecule sequence and property pairs D = ((), = 1,..., n). The log-like geelikklihood function is L(0) 1/n Di=1 log po (Z4, y4).
Since Un (zn),
write the model
po( y) = pa(2]2 Ua (20)))) (u|z
(4)
where po(20) ~ (0, Ta). The learning gradient can be calculated as follows,
Va log (1r, (r, y) (oo()))).))) Po + V (elog, For the prior model, the learning gradient
(5)
8 (z. y) Exo v) V ( + V. log (o The learning gradient for the molecule generation model is 6, y) (zol V log po Ua
The learning gradient for the predictor model is
óy (zr, v) (po V log p, (v)
Estimating these expectations requires MCMC sampling of the posterior distribution (=o v). We use Langevin dynamics (Neal, 2011). For a target distribution x T==). the dynamics iterates as follows,
-+1 = =" + vV log T2²) + v2sE",
(6)
where indexes the time step, is step size, and ~ /N Ia) is the Gaussian white noise. Here, (2)) is instantiated by the posterior distribution pe v). With (zo), we have p(zolz,)) y)
The gradient is
log logps(s (20) (x00 + so logp, (v|z) Va so log logpa(Ce (=)l2, v) = V sn log (o( + V Σ
autoregreasive molecule gencration
property prediction
We initialize =3 =0=0 (0, Ia), and employ N steps of Langevin dynamics (e.g. N 15) for approx- imate sampling from the posterior distribution, rendering our learning algorithm as an approximate MLE. See Pang al. (2020); Nijkamp al. (2020); Xie et al. (2023) for theoretical understanding of the learning algorithm based on the finite-step MCMC.
Pretrain LPT In practical applications involving multiple molecular generation tasks, each char- acterized by a different target property y, each model (s(z, y) may require separate training. To enhance efficiency, we adopt pretaining strategy focusing solely molecule sequences. In this case, we aim to maximize Σn_ ==1 log log pa (x;) = En log po (z,, 2 ))d2, The learning gradient is U. 20) After pretraining LPT, we finetune the model with target properties using Eq. (5) for a small number of epochs. This two-stage approach is adaptable for supi-supe scenarios where property values are limited.
Vo log po (z) Epe (=))=) [V log (2
3.3 Property Conditioned Generation
After MLE learning from a collected (offline) dataset, LPT is capable of generating molecules with desired properties. Since the latent prompt is designed as the information bottleneck, we can generate the molecule given property value as p(rly) p p(zoly)p(z(2a))ds We first infer the latent prompt via posterior sampling using Bayes' rule,
2o >>(eeol)) x (wlz (z(2))).
This posterior sampling performed using Langevin dynamics similar to the training process. Specifically, replace the target distribution Eq. (6) with po(zoly) and run MCMC for fixed number of steps, i.e.,
20
=
V log (ogg(=zlv)) + V28e"
where V Sg logg(<zll)) = V (logp((20) + l (og((y(20)) = 200 + o²
(v
(8)

---

This process of sampling p( ly) iteratively refines the latent prompts increasing their likelihood given the desired property or constraints. This optimization of molecules achieved in the latent space by gradient-based sampling as form of test-time computation. Once we generate the latent prompt 2, the molecule generation model uses this latent prompt to sample the next element r:(1) ~ Pa(2)()) (x((()) 2 = Un (2n)) until termination.
MLE learning of the joint model is equivalent to minimizing the DKr. (pdata (2t, y) [lpo (1. v)). Note that Eq. (7) is reliable when condition y is supported by (y). However, in real-world molecule design, desired property values often lie far from those in the collected offline dataset. For such cases, LPT should be applied in an online learning setting, as discussed in the next section.
3.4 Optimization via Online Learning with LPT
When the desired property value y is not supported within the learned distribution po x, y). propose an online learning approach to gradually shift the model distribution towards regions supporting desired properties. We assume access to the oracle functions o(x) via software such as RDKit (Landrum al.), AutoDock-GPU SSantos-Marting et al., 2021).
As discussed Sec. 3.3, generated molecules are reliable when the desired property value is supported by the learned model. To leverage this, we propose an iterative distribution shifting method, as shown in Alg. 2, as a form of online learning for LPT. In each iteration of distribution shifting, we
(a) Sample molecules from the learned LPT, using incremental extrapolation to generate a synthetic dataset with improved desired properties.
(b) Use hindsight relabeling for the sampled molecules' properties based on oracle functions.
(c) Apply MLE learning of LPT based on this synthetic dataset of molecule-propenss pairs as described in Sec. 3.2.
This process is repeated until the model converges or budget limits are reached. This online learning method maintains a latent space generative model (LPT) and a synthetic dataset and gradually shifts both towards regions of desired targets.
In step (a), to account for small extrapolation, we use property-conditione generation as mentioned in Sec. 3.3 by setting the desired property y y" + dy. where y" lies on the boundary of the learned LPT in the previous shifting iteration (e.g. y" can be the maximum value) and 5y is a small increment representing extrapolation. The hyper-parameter 6, quantifies the shifting speed. Specifically, since the latent prompts decouple the molecule generation and property prediction, first sample the latent prompts 20 Pooly y² 6yy) before the molecule generation and then use x ~ (z/z = (z0)) to generate the novel molecules. To sample molecules satisfying binary constraints, we use zo ~ (=olyy
1). This latent space sampling scheme relieves the expensive autoregressive sampling in the data space, and allows efficient gradient-based sampling in low-dimensional latent space, such as Langevin dynamics. In step (b), after generating the novel molecules in step (a), we use the oracle function o(x) to relabel them, obtaining synthetic dataset of molecule-prose propeetyrpytyyyyoooooo pairs with ground-truth values. This synthetic dataset serves as a replay buffer, storing generated molecules along the optimization trajectory, and is used for MLE training of the LPT in step (c). The illustration of the online learning of LPT is depicted in Fig. 2.
Compared to population- based methods such as genetic algorithms (Nigam et al., 2020) and particle- swarm algorithms (Winter et al., 2019), our method maintains both a synthetic dataset (which can be considered a small population) and a generative model to fit the dataset, enabling the improvement of generated molecules from the model. The model itself can be seen as infinite population, as it can generate an unlimited number of new samples.
3.5 Efficiency Analysis
In molecule design, computational efficiency and sample efficiency are crucial, with varying priorities depending on the objectives. Computational efficiency measures the ability to learn quickly with limited resources, while sample efficiency assesses the capacity to learn good model with minimal oracle function interactions. Generative models are applied to molecular design for two purposes: obtaining starting points and optimizing them to meet specific objectives. For objectives accessible via existing software, computational efficiency is prioritized, aiming to train the model and query software as quickly as possible until convergence. For properties that are time-consuming and
5

---

16
12
Figure 2: Illustration of online learning LPT. For each shift iteration, we plot the densities of docking scores E' using AutoDock-GPU. The increase of the docking scores indicates better binding affinity.
expensive to simulate, such as wet-lab experiments, sample efficiency is more important, minimizing the number of oracle queries for practical usage. In real-world scenarios, a combination of both may be necessary, depending on the specific requirements. We propose the following techniques for LPT to address efficiency issues accordingly.
Computational efficiency The computational overhead for online learning of LPT primarily arises from the iterative MCMC sampling procedure. In the posterior sampling stage of p=(zoly) and p0(20/2r, y) in step (a) and (c), rather than using Gaussian noise-initialized MCMC chain with fixed number of MCMC steps (N = 15) for each learning iteration, we employ the Persistent Markov Chain (PMC) method (Tieleman, 2008; Xie et al., 2016; Han et al., 2017; Xie et al., 2019), which amortizes sampling across shifting itcrations. Specifically, for the starting point 2j - in Eq. (6) of each MCMC chain, its initialization is drawn from Gaussian distribution the first learning iteration, then subsequently from the previous iteration's 's sampling output. With this approach, the number of MCMC updates can be reduced to N 2 steps per sampling stage, achieving approximate 5x speedup in posterior sampling.
Sample efficiency We aim to improve sample efficiency in both steps (a) and (c) in Sec. 3.4, In step (a), generation, we refine the exploration strategy during test-time computation. This approach demonstrates clear advantage of posterior inference over direct use of VAE or GAN. By guiding the learned LPT towards exploitation during generation, we encourage the latent prompts 2 to converge the modes of the posterior distribution in Langevin dynamics. This concept is analogous to the intuition behind classifier guidance in conditional diffusion models (Dhariwal and Nichol, 2021; adjust の² to balance the trade-off between exploitation and exploration: 2., log p( p(22l1/ ) = V =o (log p( 20) + log ((yp(()))) = -2 + o² (y s,
Ho and Salimans, 2022). In Eq. (8),
When 1/6" 1, the sampled latent prompts 2 represent the density of the posterior distribution, resulting in an efficient exploration scheme. As 1/o" increases, the sampled posterior concentrates around the modes of the posterior distribution, indicating increased confidence and stronger bias towards exploitation. In step (c), we leverage the synthetic dataset to train the LPT on the most informative samples in terms of property values. By training the LPT on a distribution that assigns higher probability mass to high-value points and lower mass to low-value points, the training objective encourages a larger fraction of the feasible region's volume to model high-value points while simultaneously using other data points to learn useful representations and avoid overfitting. We modify the standard objective Σ 1/z log (1,, y4) by assigning an importance weight w; to each molecule-pee molecule-property pair (I,. in the synthetic dataset, resulting in biased objective function, Σx= w; log Po (2,, y:), where Σ, w. 1. In our experiments, we set w; 1/N if y in the top-N property scores, and wi 0 otherwise. This approach is inspired by prioritized experience replay (Schaul et al., 2015) in online reinforcement learning and weighted retraining (Tripp et al., 2020) in black-box optimization, both of which prioritize learning from the most informative samples to improve sample efficiency.
4 Related Work
Latent space optimization Latent space optimization has been widely applied in generative models and dinensiinnsi data manipulation (Jin et al., 2018; Kusner et al., 2017: Kajino, 2019; Dai et al., 2018). This method involves representing data in a lower-dimensional space while retaining its essential characteristics. Latent space optimization improves the fidelity and quality of generated data and facilitates more efficient and effective exploration of the latent space. This leads to better generalization and robustness in various applications, such as image synthesis (Karras al., 2019; Razavi et al., 2019; Song et al., 2020; Dhariwal and Nichol, 2021; Rombach et al., 2022), data

---

compression (Ballé et al., 2016; Mentzer et al., 2018), molecular optimization (Kong et al., 2023; Jain al., 2023; Zhu et al., 2024), and automatic machine learning (Liu et al.. 2018; Zhang et al., 2019). When the search space is significantly simplified in the latent space, familiar Bayesian optimization (BO) tools can be readily applied (Maus et al., 2022; Tripp et al., 2020). Unlike BO-based methods, our LPT is based on the explicit probabilistic form of p(=ly) which allows us to perform optimization as conditional generation using Bayes' rule.
Generative molecule design This research follows two main approaches. The first uses latent space generative models to translate discrete molecule graphs into continuous latent vectors, enabling optimization of molecular properties within the latent space (Gómez- Boonne-ll et al., 2018; Kusner al., 2017; Jin al., 2018; Maziarz et al., 2021; Eckmann et al., 2022; Kong et al., 2023). The second directly employs combinatorial optimization methods, such as reinforcement learning, to fine-tune molecular attributes within the graph data space (You et al., 2018; De Cao and Kipf, 2018; Zhou et al., 2019; Shi et al., 2020; Luo et al., 2021; Du et al., 2022). Alternative data space methodologies, like genetic algorithms (Nigam et al., 2020), particle-swarm strategies (Winter al., 2019), Monte Carlo tree search (Yang et al., 2020), and scaffolding trees (Fu et al., 2021), have also gained traction.
Experiments
We demonstrate the effectiveness of our approach across a wide range of optimization tasks. In the context of molecule design, this includes binding affinity maximization, constrained optimization, and multi-objective optimization (see Sec. 5.2). Additionally, we optimize protein sequences for high fluorescence and DNA sequences for enhanced binding affinity (see Sec, 5.3). Finally, we validate the sample efficiency of LPT by performing optimization with a limited number of oracle function queries (see Sec. 5.4). Additional experiments, including robustness to noisy oracles, online learning from scratch, ablation studies and a detailed discussion of related works and baselines, are provided in Apps. A.2 and A.4.
5.1 Overview
Molecule Sequence Design For molecule design tasks, we use SELFIES representations of the ZINC (Irwin et al.. 2012) dataset, which comprises 250K drug-like molecules as offline dataset D. We utilize RDKit (Landrum et al.) to compute several key metrics, including penalized logP, drug-likeness (QED), and the synthetic accessibility score (SA). Additionally, we use AutoDock- GPU (Santos-Martins et al., 2021) to derive docking scores E, which serve as proxies for estimating the binding affinity of compounds three protein targets: the human estrogen receptor (ESRI), human peroxisomal acetyl-CoA acy acyltransferase (ACAAI), I), and Phosphoglyceratt dehydrogenase (PHGDH). The binding affinity is expressed as the dissociation constant K D (nM), which is approximated by the formula KD -Ece where E is the docking score and c is a constant. A lower KD value indicates stronger binding affimity. ESRI is a well-characterized protein with numerous known binders, making a suitable reference point for evaluating molccules generated by our model. In contrast, ACAAI has no known binders, providing an opportunity to test the model's capability for de novo design (Eckmann al., 2022). Additionally, we propose to design molecules that bind to PHGDH, an enzyme pivotal in the early stages of L-serine synthesis. Recently, PHGDH has gained attention as a potential therapeutic target in cancer treatment due to its involvement in various human cancers (Zhao et al., 2020). The crystal structure of PHGDH (PDB: 2G76) is well-established, and there exists a comprehensive case study on the development PHGDH inhibitors, showcasing structure-based progression from simpler to more complex molecules targeting its NAD binding site (Spillier and Frédérick, 2021), as illustrated in Fig. 3. Furthermore, we observe that the Kn values estimated from wet lab experiments align closely with the trends predicted by AutoDock-GPU, This congruence makes PHGDH-NAD an excellent case study for testing LPT on single-objective, structure-constrainee and multi-objective optimization tasks using AutoDock-GPUl More details can be found in App. A.S.
Protein and DNA Sequence Design We further apply our method to biological sequence design via two tasks in Design-Bench (Trabucco et al., 2022): TF Bind 8 and GFP. To be specific, the TF Bind task focuses on identifying DNA sequences that are 8 bases long, aiming for maximum binding affinity. This task contains a training set of 32,898 samples and includes an exact oracle function. The GFP task involves generating protein sequences of 237 amino acids that exhibit high fluorescence.
7

---

SPR 1.6 uM
SPR - -0.18 Autodock 0.97
36.1 JM
Figure 3: Illustration of the development of PHGDH inhibitors (Spillier and Frédérick, 2021). Surface Plasmon Resonance (SPR) and AutoDock KD values are reported for each inhibitor. The trends observed between the experimental SPR values and the computational AutoDock values align well, validating the computational approach.
For this task, outlined in Trabucco et al. (2022). Due to the unavailability an exact oracle function for the GFP task, follow the same oracle function preparation regression model on the full dataset with a total of 56,086 samples as the oracle function. We evaluate the design performance and diversity of the generated samples.
subset of 5,000 samples the training set by following the methodology
in Design-Bench and train Transformer
Training Setup The prior model, Pa (z), of LPT is a one-dimensional UNet (Ronneberger et al., 2015) where contains 4 tokens, each of size 256. The sequence generation model, p=(=))=, is implemented as a 3-layer causal Transformer, while a 3-layer MLP serves as the predictor model, P, (g)2). As described in Sec. 3.2, pre-train LPT on molecules for 30 epochs and then fine-tune with target properties for an additional 10 epochs, following the procedure outlined Alg. in App. A.3. We perform up 25 iterations of online learning, generating 2,500 samples per iteration, which totals a maximum of 62.5K oracle function queries. We use the Adam W optimizer (Loshchilov and Hutter, 2019; Kingma and Ba, 2014) with weight decay of 0.1. Training was conducted NVIDIA A6000 GPU, requiring 20 hours for pre-training, 10 hours for fine-tuning, and 12 hours for online learning. Additional details can be found in App. A.3.
5.2 Binding Affinity Maximization
Single-Objectivy Optimization For the single-objective binding affinity optimization task, we aim to design ligands with optimal binding affinitics to ESRI, ACAAI, and PHGDH de novo design, without any constraints. LPT does not use any prior knowledge of cxisting binders, and exclusively the crystal structures of the aforementioned proteins. The predictor model (yl2)) for this task a regression model that estimates docking scores. We compare our model with several baseline methods, which introduced in App. A.4. As shown in Tabs. 1 and 2, our model significantly surpasses other methods across all three binding affinity maximization tasks in terms of K often achieving substantial improvement. Lower K n values indicate better performance. Furthermore, in Tab. 2, we report the average performance of the top 50 and top 100 molecules to demonstrate that model effectively generate diverse pool of candidate molecules with the desired properties. Visualizations of generated molecules are provided in App. A.
Table 1: Single-objective binding affinity optimiza- tion results for ESRI and ACAAI. Top per-
Table 2: Single-objective binding affinity maxi- mization results PHGDH, reporting the 1st, 2nd, and 3rd performance, along with average performance of top 50 and top 100 molecules for each model. Performance is measured by KD1 2nM.. The highest scores are in bold.
formance in terms of K (nM) achieved by each model are reported. The best scores are in bold.
ESRI Kp (L)
ACAAI Kp (1)
METHOD
1ST
2RD
3RD
1sr
2RD
3RD
GCPN
6.4
6.6
8.5
75
83
84
PHGDH Ko (1) SGDS
MOLDQN
373
1062
337
LIMO
I.PT
MARS
25
47
51
370
520
590
GRAPHDF
17
64
69
163
203
236
1ST
13.87
3.65
0.07
LIMO
0.72
0.89
37
2ND
18.79
6.59
0.16
SGDS
0.03
0.03
0.04
0.11
0.11
0.12
3RD
21.15
7.16
0.19
ToP-50
81.16
34.41
14.7
1 5.43
0.95
0.46
LPT
0.004
0.005
0.009
0J037
0.041
0.045
TOP-100
125.06
52.59
24.5
4 14.0
1.66
0.82
Straeture-conssrrained Optimization The structure-constraiind optimization task mimics lead optimization in drug discovery, aiming decorate fixed core substructure to optimize activity and pharmacological properties. Our model's factorization, p(2, y) = p(=))(x)==)p(/1)) enables the decoupling of molecule generation and property prediction, simplifying conditional generation.

---

MBlaM
Admsaa AM
KO-LITAM
7M5
(a)
Figure 4: (a) crructure-cossraiing Optimization. Conditionally generated compounds C2 and C3 closely resemble the human- semaandesigned compounds C2 and C3 shown in Fig. 3. Additionally, the right column also presents further optimized compounds that achieve improved Ko scores. (b) Illustration of generated molecules binding PHGDH with docking poses generated by AutoDock-GPP The left panel visualizes the molecule generated through multi-objective optimization, while the right panel displays the molecule generated via structure-constrrained optimization.
Given a substructure = (xx(1) ,...,, (1)),, (M)), we aim to sample from po(z, y/.). This is accomplished by sampling p(=ly) and then x p(æ+ z), which only requires rearranging é's sequence to start from the desired atom. In Fig. 3, Compound (C2) is designed by humans as an extension of Compound (C1), and Compound 3 (C3) is similarly an extension of C2. In Fig. 4, we show that: (I) given CI or C2, our model is able to design compounds similar to the human-designed C2 or C3, and (2) model can identify molecules that outperform those designed by humans. Mcanwhile, confirm that C2, C3, and LPT-generated molecules are novel compared to the ZINC training set, with average Tanimoto similarity less than 0.5. More generated molecules are shown in App. A.6.2.
Our model explores high-affinity PHGDH inhibitors by adding functional groups to an indole backbone, a common scaffold for such inhibitors (Spillier and Frédérick, 2021). The model identifies aromatic or heteroaromatic groups, such as benzene or pyridine, at the second position as frequently occurring and exhibiting higher binding scores compared to the indole backbone itself. This finding aligns with reported data: a published molecule with benzothiophene backbone similar CI exhibits binding affinity of 470 μ.M, while C2, which includes aromatic group the second position, shows a significantly improved binding affinity of 1.6 μM (Spillier and Frédérick, 2021; Fuller et al., 2016). Furthermore, the model introduces second functional group at the sixth position of the indole in C2, generating inhibitors closely resembling the structure of C3. The observed trend of increasing binding affinities (1.6 μM for C2 and 0.18 μM for C3) aligns with the literature values, providing validation for our proposed method in identifying potential high-affinity inhibitors.
Multi-Objective Optimization For multi-objeccive optimization tasks, we aim simultaneously maximize binding affinity and QED, while minimizing SA. These objectives are balanced as weighted combination, with constraints of QED 0.4 and SA < 5.5. We evaluate our method on three protein targets: ESRI, ACAAI, and PHGDH, comparing the results against two baseline methods, LIMO (Eckmann et al., 2022) and SGDS (Kong et al., 2023). As shown in Tab. 3, our method, LPT, achieves QED and SA scores comparable to those of SGDS while significantly improving binding affinity across all three protein targets, demonstrating its superior modeling capability. Examples the generated molecules are provided in App. A.6.1.
Table 3: Multi-objective optimization Results. Top 2 performance, measured by Kp (nM), QED and SA, reported for each method. Baseline methods include LIMO (Eckmann et al., 2022) and SGDS (Kong et al., 2023). Best results are marked in bold, and the second best results are underlined.
ESRI QHI
ACAAI QED
PHGDH QED>
LIGAND
Kp J
Ko
SA
Kp
J
LIMO 19
4.6
0.43
0.57
5.5
29.15
0.33
4.73
LIMO n
2.8
0.64
4.9
31
0.44
4.9
42.98
0.20
5.32
SGDS 1
0.36
0.44
3.99
4.55
0.56
4.07
4.47
0.54
3.37
SGDS ndd
1.28
0.60
4.58
0.42
4.02
LPT 1MT
0.04
0.58
3.46
0.18
0.50
4.85
0.02
0.50
3.11
LPT 2
0.05
0.46
3.24
0.21
0.61
4.18
0.03
0.43
3.22

---

5.3 Biological Sequence Design
Our model excels in biological sequence design, single-objective optimization application, as demonstrated by two benchmarks in Design- Bench (Trabucco et al., 2022): TF Bind 8 and GFP. Tab. 4 shows that LPT significantly outperforms other methods in these tasks. In the TF Bind 8 task, our approach surpasses the strong competitor GFlowNet-Al. (Jain et al., 2022), while maintaining comparable diversity. For the GFP task, we achieve superior performance with reasonable diversity.
Table 4: Results of biological sequence design on TF Bind and GFP benchmarks. Performance and diversity are evaluated on 128 samples. Results of other baselines are obtained from Jain et al. (2022). Bold highlighting indicates top scores.
TF BIND
GFP
METHOD
PERFORMANCE
DIVERSITY
PERFORMANCE
DIVERSITY
DYNAPPO
0.58 + 0.02
5.18 + 0.04
0.05 0.008
12.54 + 1.34
COMs
0.74 1 0.04
4.36 0.24
0.831 1 0,003
8.57 1: 1.21
BO-QEI
0.44 = 0.05
4.78 1 0.17
0.045 = 0.003
12.87 1.09
CBAS
0.45 -.45=0.14 0.14
0.817 0.012
8.53 +: 0.65
MINS
0.40
5.57
8.31 0.02
EMA-ES
0.47 + 0.12
4.89 0.01
0.063 1 0.003
10.52 + 4.24
AMORTIZEDBO
0.62 + 0.01
4.97 + 0.06
0.051 + 0.001
16.14 2.14
GFL
0.84 1 0.05
4.53 0.46
0.05 1 0.010
21.57
1 3.73
LPT
0.954 + 0.002
4.58 0.06
0.857
0.003
9.45 + 0.23
5.4 Sample Efficiency
We validate LPT's sample efficiency on the Practical Molecular Optimization (PMO) benchmark (Gao et al., 2022). where
objectives (MPO) are optimized within limited oracle budget of 10K queries. Tab. 5 shows that our method, LPT, surpasses previous approaches, such as MARS (Xie et al., 2021), GFlowNet (Jain et al., 2022) and SMILES/SELFIE VAE (o6mez-Bombarelle et al., 2018; Maus al., 2022) with Bayesian Optimization (BO). Also, LPT achieves comparable per- formance to LSTM HC (Brown et al., 2019). the best generative molecule design method in PMO, and demonstrates performance on par with GP-BO (Tripp et al., 2021), the best BO-based method in PMO, under the limited budge of oracle function queries. We acknowledge that there remains a performance gap between generative model-based optimization and methods like genetic algorithms when working with a small budget. This is primarily due to the data-intensive requirements of training generative models. To ensure fair assessment, generative molecule design methods within PMO. It's worth noting that generative models offer distinct advantages when maintaining relatively large budgets, as the learned model itself can be viewed as infinite populations for further exploration.
comparison focuses representative
Table 5: Comparison of sample efficiency on the PMO benchmark. The mean and standard deviation of AUC Top-10 10 from 5 independent runs are reported. Best results are marked in bold.
METHOD
AMLUDIPINE
FEXOFENADINE
OSIMERTIINIS
PURINDOPRIL
KANOLAZINE
ZALEPLON
SUM
GFLOWNET
0.444 1: 0.004
0.693
0.005
0.784
0.001
0.430
4: 0.010
0.652
0.002
0.035 4: 0.030
3.038
0.504
0.016
0.711
0.777
0.006
0.462
0.006
+ 0.046
3.381
LSTM HC
0.593 + 0.016
0.725
0.003
0.796
:: 0.002
0.489
:: 0.007
0.714
0.008
0.205 +: 0.006
3.523
SMILES- VAE BO
0,533 + 0.009
0.671
0,003
0.771
: 0.002
0.442
4 0.004
0.457
0.012
0.039
: 0.012
2,913
SELFIES-VAE
0.005
0.670
0.429
0.025
3.038
GP-BO
0.583
0.044
0.722
0.005
0.787
1 0.006
0.493
4: 0.011
0.735
0.013
0.221
4: 0.072
3.541
LPT
0.608 : 0,005
0.714
0,003
0.784
0,011
0,511 : 0.002
0.682
0.007
0.245 : 0,003
3.544
Limitation and Conclusion
In this work, we presented LPT, novel generative model for molecule design that achieves strong performance through its offline and online learning algorithms. Contemporary work has extended the similar model to offline reinforcement learning (Kong et al., 2024). While the model demonstrates significant potential, there are opportunities to better understand how LPT handles the inherent trade-offs in multi-objective optimization scenarios, particularly in characterizing the Pareto front nature of optimal solutions. Future work could also explore alternative architectures to extend LPT's applicability beyond sequence-baar based optimization problems in science and engineering.
10

---

Acknowledgements
The authors would like to thank the amonymous reviewers for providing constructive comments and suggestions to improve the work. The work was partially supported by NSF DMS-2015577, NSF DMS-2415226, a gift fund from Amazon, and XSEDE grant CIS210052.
References
Christof Angermueller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy Colwell. Model-based reinforcement learning for biological sequence design. In International conference on learning representations (ICLR), 2019.
Johannes Ballé, Valero Laparra, and Eero P Simoncelli. End-to-end optimized image compression. arXiv preprint arXiv:1611.07700 2016.
David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International conference machine learning (ICML), pages 773-782, 2019.
Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of Chemical Information and Modeling. 59(3): 1096-1108, 2019.
Austin H Cheng, Andy Cai, Santiago Miret, Gustavo Malkomes, Mariano Phiclipp, and Alán Aspuru- Guzik. Group selfies: a robust fragment- based molecular string representation. Digital Discovery, 2023.
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. autoencoder for structured data. International Conference on Learning Representations (ICLR), 2018.
variational
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs. arXiv preprint 1805: 11973, 2018.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, pages 8780-8794, 2021.
Yuanqi Du, Tianfan Fu, Jimeng Sun, and Shengchao Liu. Molgensurvey: A systematic survey in machine learning models for molecule design. arXiv preprint arXiv:2203. /4500, 2022.
Peter Eckmann, Kunyang Sun, Bo Zhao, Mudong Feng, Michael K Gilson, and Rose Yu. Limo: Latent inceptionism for targeted molecule generation. In International Conference on Machine Learning (ICML), 2022.
Clara Fannjiang and Jennifer Listgarten. Autofocused oracles for model design. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 19945-12956, 2020.
Tianfan Fu, Wenhao Gao, Cao Xiao, Jacob Yasonik, Connor W Coley, and Jimeng Sun. Differentiable scaffolding tree for molecular optimization. arXiv preprint arXiv:2109.10669 2021.
Nathan Fuller, Loredana Spadola, Scott Cowen, Joe Patel, Heike Schönherr, Qing Cao, Andrew McKenzie, Fredrik Edfeldt, Al Rabow, and Robert Goodnow. An improved model for fragment- based lead generation astrazeneca. Drug Discovery Today, 21(8): 2016.
Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor Coley. Sample efficiency matters: benchmark for practical molecular optimization. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 21342-21357, 2022.
Rafael BómerzlBo Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobate Benjamín Sánchez- Maancheez-engeling. Dennis Sheberla. Jorge Ryan P Adams, and Alán Aspuru-Guzik, Automatic chemical design using data-driven continuous representation of molecules. ACS Central Science, 4(2):268-276, 2018.
Timothy D Hirzel,
Tian Han, Yang Lu, Song-Chun 7hu, and Ying Nian Wu. Alternating brok-propag for generator network. In AAAI Conference Artificial Intelligence (AAAI), pages 1976-1984, 2017.

---

Nikolaus Hansen. The ema evolution strategy: a comparing review. Towards a new evolutionary computation: Advances in the estimation of distribution algorithms, pages 75-102, 2006.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598. 2022.
John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan G Coleman. Zinc: free tool discover chemistry for biology. Journal of Chemical Information and Modeling. 52(7): 1757-1768, 2012.
Moksh Jain, Emmanuel Bengio, Alex Hermandez-Garcin Jarrid Brotor-Brooos,.. Bonaventure FP Dossou, Chanakya Ajit Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, et al. Biological sequence design with gflownets. In International Conference Machine Learning (ICML), pages 9786-9801. PMLR, 2022.
Moksh Jain, Sharath Chandra Raparthy, Alex Hernández-Garcia, Jarrid Rector-Brooks, Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-objective gflownets. In International Conference on Machine Learning (ICML.) pages 14631-14653 2023.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International Conference Machine Leurning (ICML), pages 2323-2332, 2018
Hiroshi Kajino. Molecular hypergraph grammar with its application to molecular optimization. In International Conference Machine Learning (ICML), pages 3183-3191, 2019.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4401-4410, 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
Diederík Kingma and Max Welling. Auto-encoding variational bayes. International Conference on Learning Representations (ICLR), 2014.
Deqian Kong, Bo Pang, Tian Han, and Ying Nian Wu. Molecule design by latent space mergy-based modeling and gradual distribution shifting. Conference on Uncertainty in Artificial Intelligence (UAT), volume 216, pages 1109-1120, 2023.
Deqian Kong, Dehong Xu, Minglu Zhao, Bo Pang, Jianwen Xie, Andrew Lizarraga, Yuhao Huang, Sirui Xie, and Ying Nian Wu. Latent plan transformer for trajectory abstraction: Planning as latent space inference. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alan Aspuru-Guzik. Self- referencing embedded strings (selfies): A 100% robust molecular string representation. Machine Learning: Science and Technology, 1(4):045024, 2020.
Matt J Kusner, Brooks Paige, and José Miguel Hernández-Lobato. Grammar variational autoencoder. In International Conference on Machine Learning (ICML), pages 1945-1954, 2017.
Greg Landrum et al. RDKit: Open-source cheminformatics. URL https:/ //ww:://wwwwwwwo rwkit. rdwit.org....
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv
preprint arXiv: arXiv:1806. 00008800000005 2018.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations (ICLR), 2019.
Youzhi Luo, Keqiang Yan, and Shuiwang Ji. Graphdf: discrete flow model for molecular graph generation. In International Conference Machine Learning (ICML), pages 7192-7203, 2021.
Natalie Maus, Haydn Jones, Juston Moore, Matt J Kusner, John Bradshaw, and Jacob Gardner. Local latent space bayesian optimization over structured inputs. In Advances Neural Information Processing Systems (NeurIPS), volume 35, pages -4505-34518, 2022.
12

---

Krzysziof Maziarz, Henry Jackson-Fing Pashmina Cameron, Finton Sirockin, Nadine Schneider, Nikolaus Stiefl, Marwin Segler, and Mare Brockschmidt. Learning to extend molecular scaffolds with structural motifs. arXiv preprint arXiv: 103. 038643 2021.
Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. Condi- tional probability models for deep image compression. In JEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4402-4402, 2018.
Edouard Mullarky, Jiayi Xu, Anita D Robin, David Huggins, Andy Jennings, Naoyoshi Noguchi, Andrea Olland, Damodharan Lakshminarasimhan. Michael Miller, Daisuke Tomita, et al. Inhibition of 3- 3-phosphoglyeerat dehydrogenase (phgdh) by indole amides abrogates de novo serine synthesis in cancer cells. Bioorganic Medicinal Chemistry Letters, 2911))2550325 2510,, 2019.
Radford M Neal. MCMC using hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, 2011.
AkshatKumar Nigam, Pascal Friederich, Mario Krenn, and Alán Aspura-Guzik. Augmenting genetic algorithms with deep neural networks for exploring the chemical space. In International Conference on Learning Representations (ICIR), 2020.
Erik Nijkamp, Bo Pang. Tian Han, Linqi Zhou, Song-Chun Zhu, and Ying Nian Wu. Learning multi-layer latent variable model via variational optimization of short run meme for approximate inference. In European Conference on Computer Vision (ECCV), pages 361-378, 2020.
Bo Pang, Tian Han, Erik Nijkamp, Song-Chun Zhu, and Ying Nian Wu. Learning latent space energy-based prior model. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer- Assisted Intervention (MICCAI), pages 234-241, 2015.
Debangshu Samanta and Gregg I. Semenza. Serine synthesis helps hypoxic cancer stem cells regulate redox. Cancer Research, 76(22):6458 6462, 2016.
Diogo Santos- Leonardo Solis Vasquez, Andreas F Tillack, Michel F Sanner, Andreas Koch. and Stefano Forli. Accelerating AutoDock4 with GPUs and gradient-based based local search. Journal of Chemical Theory and Computation, 17(2):000--0733, 2021.
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint 1eXiv:1511.05000 2015.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. arXiv preprint XXiv:2001.09382. 2020.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020.
Quentin Spillier and Raphaël Frédérick. Phosphoglycerate dehydrogenase (phgdh) inhibitors: A comprehensive review 2015-2020. Expert Opinion on Therapeutic Patents, 31(7):597-608, 2021.
Kevin Swersky, Yulia Rubanova, David Dohan, and Kevin Murphy. Amortized bayesian optimization over discrete spaces. In Conference on Uncertainty in Artificial Intelligence (UAI), pages 769-778, 2020.
Tijmen Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In International Conference Machine Learning (ICML), pages 1064-1071, 2008
13

---

Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. In International Conference on Machine Learning (ICML., pages 10358-10365 2021.
Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Design-benche Benchmarks for data-driven offline model-based optimization. In International Conference on Machine Learning (ICML), pages 16658-166777 2022.
Austin Tripp. Erik Daxberger, and José Miguel Hermández-Lobato, Samplc-efficient optimization in the latent space of deep generative models via weighted retraining. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 1 11259- 1272, 2020.
Austin Tripp, Gregor NC. Simm, and José Miguel Hernández-LLobtto A fresh look at de novo molecular design benchmarks. In NeurIPS 2021 AI for Science Workshop. 2021.
Judith E Unterlass, Robert Wood, Arnaud Baslé, Julie Tucker, Céline Cano, Martin ME Noble, and Nicola Curtin. Structural insights into the enzymatic activity and potential substrate promiscuity of human 3-phosphoglycene
dehydrogenase (phgdh). Oncotarget, 8(61): (:61):10440 104478, 2017.
David Weininger. Smiles, a chemical language and information system. 1. introduction to methodol- ogy and encoding rules. Journal of Chemical Information and Computer Sciences, 28(1):31-36, 1988.
James T Wilson, Riccardo Moriconi, Frank Hutter, and Marc Peter Deisenroth. The reparameterization trick for acquisition functions. arXiv preprint arXiv:1712.00422 2017.
Robin Winter, Floriane Montanari, Andreas Steffen, Hans Briem, Frank Noé, and Djork-Arné Clevert. Efficient multi-objective molecular optimization in a continuous latent space. Chemical Science, (0034):8016-8022 2019.
Jianwen Xie, Yang Lu, Song-Chun Zhu, and Ying Nian Wu. A theory of generative convnet. In International Conference Machine Learning (ICML), pages 2635-2644, 2016.
Jianwen Xie, Ruiqi Gao, Zilong Zheng, Cong-Chun Zhu, and Ying Nian Wu. Learning dynamic generator model by alternating back proo-propagttion through time. In AAAI Conference on Artificial Intelligence, (AAAT), pages 5498 00088-55777740 2019.
Jianwen Xie, Yaxuan Zhu, Yifei Xu, Dingcheng Li, and Ping Li. A tale of two latent flows: Learning latent space normalizing flow with short-run langevin flow for approximate inference. In AAAI Conference on Artificial Intelligence (AAAI), 2023.
Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. In International Conference on Learning Representations (ICLR), 2021.
Xiufeng Yang, Tanuj Kr Aasawat, and Kazuki Yoshizoe, Practical massively parallel monte-carlo tree search applied to molecular design. arXiv preprint arXiv:2006. 10504, 2020.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems (NeurIPS), pages 6410-6421, 2018.
Muhan Zhang, Shali Jiang, Zhicheng Cui, Roman Garnett, and Yixin Chen. D-vae: A variational autoencoder for directed acyclic graphs. In Advances in Neural Information Processing Systems (NeurlPS), pages 1588-1600, 2019.
Xiaoya Zhao, Jianfei Fu, Jinlin Du, and Wenxia Xu. The role of d pospohosphoolyccrrtte dehydrogenase in cancer. International Journal of Biological Sciences, 16(9):1495, 2020.
Zhenpeng Zhou, Steven Kearnes, Li Li, Richard Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific Reports, 91)::-10, 2019.
Yiheng Zhu, Jialu Wu, Chaowen Hu, Jiahuan Yan. Tingjun Hou, Jian Wu, al. Sample-efficient multi- objective molecular optimization with gflownets. Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2024.
14

---

A Appendix
A.1 Model Learning
We provide the derivation of Eq. (5) in Sec. 3.2. Vepo(z,
Ve log (a v) pr( v)
po(a, v) / Vop0(1,,,2= Un(=o))
(a(a, y: Ua(2o)) V =
po (2
po po xr y) V Po y, 2 U (z(2))22
(zal=xz))))
V 2z 2 (z))))
(eal
V log
+ Vo Vologr + Vo log
V logpo + Log
A.2 Additional Experiments
A.2.1 Model Sanity Check: Penalized logP and QED Maximization
The experiments focus optimizing the Penalized logP (P-logP) and QED properties, both of which can be calculated using RDKit. Since P-logP scores are positively correlated with the length of a molecule, maximize P-logP while limiting molecule length to the maximum length of molecules in ZINC using the SELFIES (Krenn et al., 2020; Cheng et al., 2023) representation, following Eckmann al. (2022). We compare our model with several baseline methods, including JT-VAE (Jin al., 2018). MolDQN (Zhou et al., 2019), LIMO (Eckmann et al., 2022), GCPN (You et al., 2018). 8), GraphDF (You et al., 2018), MARS (Xie al., 2021), SGDS (Kong et al., 2023). Tab. 6 presents the results, demonstrating that LPT outperforms other methods and achieves the highest QED score among methods that limits molecule length.
Table 6: Results of P-logP and QED maximization. The top highest scores achieved by each model are reported. "Length Limit" indicates the application of maximum molecule length limit.
PENALIZED LOGP (+)
QED (t)
METHOD
LENGTH LIMIT
1ST
2RD
3RD
1ST
2RD
3RD
JT-VAE
X
5.30
4.93
4.49
0.925
0.911
0.910
GCPN
7.98
7.85
7.80
0,948
0.947
0.946
MOLDQN
11.8
11.8
11.8
0.948
0.943
0.943
MARS
45.0
44.3
43.8
0.948
GRAPHDF
13.7
13.2
0.948
0.948
LIMO
10.5
9.69
9.60
0,946
0.945
SGDS
26.4
25.8
25.5
0.948
0.948
0.948
LPT
38.95
38.29
38.25
0.948
0.948
0.948
A.2.2 Robustness to Noisy Oracles
To evaluate the robustness of our model in single-objective QED optimization tasks under the oracle query budget 25K, we conduct experiments with varying levels of oracle noise. define noised oracles Vnoise Vime + e, where ~ (100 o²) and varies as a percentage of the property range. The minimal degradation in performance seen Tab. demonstrates that our model resilient the noised oracles.
A.2.3 Online Learning from Scratch
We investigate LPT's performance without offline pre-training data, relying solely on online learning with budget limit of 300K (comparable to the size of the ZINC dataset plus our previous online
15

---

Table Performance across different Oracle noise levels
ORACLE NOISE
1ST
2ND
3RD
ToP-50
NONB
0.948
0.947
0.947
0.940=10003
1 %
0.947
0.947
0.946
0.939+0.004
5
0.946
0.946
0.944
0.936:1.000 0.005
10%
0.945
0.945
0.942
0.93211.006
learning budget). Results in Tab. show that LPT can effectively discover high-binding molecules for ESRI and ACAAI, even without pre-training. However, performance on PHGDH remained suboptimal compared the version with pre-training. indicating that this target may require additional oracle queries due to its inherent complexity. These findings highlight the potential of pure online learning approaches for future exploration.
Table 8: Results of online learning from scratch. Top 3 scores of K (n(M)) are reported.
1ST
2ND
3RD
ESRI
0.012
0.019
0.021
ACAAI
0.047
0.062
0.128
PHGDH
0.158
0.339
0.342
A.2.4 Ablation Studies
We conduct ablation studies to investigate the contributions of key components in LPT using challenging PHGDH single-objective optimization task. The variations our experiments included:
1. Using samples from (=) instead of pa(z)y) to generate the proposals pt.
2. Removing weighted retraining and applying the standard objective Σi 1/n log pa (2i, W:).
3. Setting the total number of shifting iterations as 1.
4. Replacing the Unet prior with a Gaussian (N, La).
As shown in Tab. 9, each component was essential, and removing or altering any them might lead to performance degradation, underscoring their importance in achieving the model's high efficacy.
Table 9: Ablation of Key Components. We report the mean and standard deviation of K (nM) over the top 100 unique molecules generated in the last shifting iteration.
METHOD
ToP 100
LPT
0.08 + 0.04
SAMPLING FROM (z(
403.18 246.54
STANDARD OBJECTIVE E 1/4 log (o
365.35 + 144.94
NUMBER OF SHIFTING ITERATION AS
41.55 + 19.02
WITHOUT LEARNABLE PRIOR
417.22 249.57
In addition, we investigate the effect of the exploitation scheme 1/0² in Eq. (8). Experiments are conducted the practical molecular optimization (PMO) benchmark, focusing optimizing the multi-property objective (MPO) for amlodipine under a 10K-query oracle budget. As shown in Tab. 10, increasing 1/22 enhances LPT's performance, indicating stronger bias toward exploiting the sampled posterior 2 and leading to improved optimization efficiency.
Furthermore, we study the effect of oracle budget size on performance, conducting experiments on single-objective ESRI binding affinity optimization task. As shown Tab. 11, LPT demonstrates robust performance even under limited oracle query budget of 10k, outperforming most existing methods. As the budget increases, I.PT continues exhibit significant performance enhancements, further highlighting its efficiency and scalability.
16

---

Table 10: Effects of the exploration schemes. We report the AUC Top-10 for the multi- pooliip objective (MPO) on amlodipine in the PMO benchmark.
EXPLORATION SCHEME 1/62
AUC Top-10
0.529
0.536
20
0.354
40
0.583
80
0.608
Table 11: Single-objective ESR1 binding affinity K (n(M) optimization with different budgets.
BUDGETS
1ST
2ND
3RD
10k
4.5
4.8
5.5
30K
1.4
1.7
2.1
50K
0.08
0.10
0.11
62.5K
0.04
0.05
0.06
A.3 Model Architecture and Training Details
As illustrated in Fig. 1, the prior model in LPT is parameterized by ID Unet, with sequence length of 4, where each of them is size of 256. The molecule generation model, p=(x]2), employs a 3-layer causal Transformer with embedding size of 256 and maximum input token length of 73. The predictor model a 3-layer MLP, which takes 2 as input and outputs predicted property values classification results. The total number of parameters for LPT is 4.33M.
LPT trained in a two-step process. Initially, it undergoes pre-training solely on molecules for 30 epochs, using cross-entropy loss with a learning rate that varies between 7.5e-4 and 7.5e-5. following a cosine scheduling approach. Subsequently, LPT is fine-tuned for 10 epochs on both molecules and their properties, outlined in Alg. with the learning rate adjusted between 3e-4 and 7.5e-5, For multi-objective optimization, i.c., simultaneously maximizing binding affinity, QED, and minimizing SA, the predictors for binding affinity and QED/SA selected as regression model and classifier, which are supervised by mean squared error (MSE) and binary cross-entropy (BCE) loss functions, respectively.
Algorithm MLE learning of Latent Prompt Transformer (LPT)
Input: Number of learning iterations T, initial parameters 0n = (ao, 3o, m)) observed sam- ples D {Ix,, Yi in F==1 posterior sampling step size 8, the number of MCMC steps N, and the learning rate 7Jo 1h, 72-
Output: 0T
for t to too 7 do
1. Postterir sampling: For each (Ir v.), sample zo pe, (=o (r; yi) using Eq. (6), where the target distribution is Pa, (zal yi) with N steps and step size
2.Learn prior model pa (2), generation model pa (z|2) and predictor model p., (+(y))):
0141 a; + no0
Σ, 8a (a v4):
3.1 B. + m
(2,, (Z,. 9:):
7t-1 = To + 72
Σ 8,, (1,, y/) as in Sec, 3.2.
end for
For LPT's online learning, as detailed in Alg. 2, we establish maximum number of shifting iterations be 25, generating 2,500 new samples in each iteration. This results in a total of up to 62.5k oracle function queries. Throughout the training processes, we utilize the AdamW optimizer (Loshchilov and Hutter, 2019) with a weight decay of O. The pre-training, fine-tuning, and online learning phases of LPT require approximately 20, 10, and 12 hours, respectively, on single NVIDIA A6000 GPU.
17

---

Algorithm 2 Online learning of LPT
Input: Number of proposals m, fixed increment oy, number of PMC steps N, initial parame-
ters 0, = (ay, Se, 72), initial dataset D = { y 1i=1 oracle function o(x), maximum number of shifting iterations T.
Output: 8_, D
while < T do
step (a): Sample molecules and properties from the learned model {z), m of H where
2¹ ~ 1 + 6y); x² ~ Pa z)).
-
step (b): Relabel the proposal property values by oracle function query: vf < o(x)), and update
the dataset by D' {24, af. v h= UD'
step (c): Update LPT using maximum likelihood on synthetic dataset D' by following Alg. 1
end while
A.4 Baselines
Our model is based on Kong et al. (2023). The differences are as follows. (1) While Kong al. (2023) used an LSTM model for molecule generation, we adopt a more expressive causal Transformer model for generation, with the latent vector serving as latent prompt. (2) While Kong et al. (2023) used a latent space energy-bace -ased model for the prior distribution of the latent vector, we assume that the latent 2 generated by a UNet transformation of Gaussian white noise vector. This approach allows us to eliminate the need for Langevin dynamics in prior sampling during training, thus simplifying the learning algorithm. (3) Our experimental results are significantly stronger, surpassing those of Kong et al. (2023) and achieving new state-of-the-arr performance.
For molecule generation, JT-VAE (Jin al., 2018) utilizes variational autoencoder (VAE). GCPN (You et al., 2018) and GraphDF (Luo et al., 2021) employ the deep graph model to dis- cover novel molecules. A reinforcement learning framework that fuses chemical domain knowledge with double Q-learning and randomized value functions for molecule optimization was presented by MolDQN (Zhou et al., 2019). MARS (Xie al., 2021) develops Markov molecular Sampling framework targeting multi-objective drug discovery, while LIMO (Eckmann et al., 2022) uses VAE-generated latent space along with neural networks for property prediction, enabling efficient gradient-based optimization of molecular properties.
In contrast to existing latent space generative models (Gómez-Bombarell et al., 2018; Kusner et al., 2017; Jin et al., 2018; Eckmann et al., 2022), our approach incorporates a learnable prior model, enabling our model effectively catch up with the evolving dataset in the optimization process.
The landscape of biological sequence design is shaped by a diverse range of computational strategies. DyNAPPO (Angermueller et al., 2019) harnesses active learning with reinforcement learning to facilitate iterative sequence generation. GFlowNet-AL (Jain et al., 2022) capitalizes on GFlowNets for generative active learning within sequence contexts. Model-based optimization techniques like COMs (Trabucco et al., 2021) and AmortizedBO (Swersky et al., 2020) integrate Bayesian Optimization with reinforcement learning, enhancing search efficiency. BO-QEI (Wilson et 2017), a variant of Bayesian optimization, refines the search process using quantile Expected Improvement. Deep generative models, e.g., CBAs (Fannjiang and Listgarten, 2020) and MINs (Brookes al., 2019), leverage deep learning for complex pattern discovery. Meanwhile, CMA-ES (Hansen, 2006) excels in -igh-dimessional optimization, adapting search strategies over generations.
A.5 Background of PHGDH and its NAD binding site
Phosphoglycerate dehydrogenase (PHGDH) an enzyme that plays crucial role in the early stages of L-serine synthesis. Recently, PHGDH has been identified as an attractive therapeutic target in cancer therapy due to its involvement in various human cancers, including breast cancer, melanoma, lung cancer, pancreatic cancer, and kidney cancer. Several studies have been devoted to the exploration of small molecule inhibitors targeting PHGDH including CBR-5884 (IC50 = 33 (uM)), NCT-503 (IC50= 2.5 (μM)), AZ PHGDH inhibitor (IC50= 180 (nM)). RAZE PHGDH inhibitor (IC50 0.01 ~ 1.5 (μM)), PKUMDL-WQ
(IC50 = 35.7 (uM)), BI-4924 (IC50 =
(nM)), and others. Additionally, the crystal structure of PHGDH (PDB: 2G76) has been elucidated.
18

---

This makes PHGDH an excellent protein model for conducting docking calculations, aiding in the precise localization of binding sites in order to optimize our algorithms.
PHGDH utilizes nicotinamide adenine dinucleotide (oxidized form, NAD+; reduced form, NADH) co-factor for enzymatic activity, producing NADH during the synthesis of 3- phosphohydroxypyyuaaar (3PHP) from 3-phosphoglycerate (3PG) (Samanta and Semenza, 2016).
The co-crystallization of PHGDH with NAD has been documented (PDB: SN6C), The NAD- pocket is surrounded by hydrophobic residues P176, Y174, L15I, L193, L216, T213, T207 and L210 (Mullarky et al., 2019). Specifically, nicotinamide moiety exhibited interactions with the protein backbone (specifically A285 and C233) as well as the side chain of D259. Additionally, the hydroxyl groups of the sugar moieties were also involved in hydrogen bonds with both the protein backbone (T206) and the side chain of D174. The phosphate linker demonstrated interactions with the main chain of RI 154 and I155, along with the side chain of R154 (Fig. 5) (Unterlass al., 2017).
T206
C233
D259
A285
D174
T
1155
R154
Figure PHGDH with NAD binding site.
A.6 Visualization of Generated Molecules
We present visualizations of subset of generated molecules from both single-objective and multi- objective optimization for ESR1, ACAA1, and PHGDH. Additionally, several conditionally generated compounds, C2 and C3, which have similar characteristics ousanndesigned ones, are also presented in Fig. 12 and Fig. 13.
A.6.1 Multi-Ohjective and Single-Objective Optimization
Figs. 6 to visualize some molecules produced during the multi-objective optimization for ESRI, ACAAI, and PHGDH, respectively. Meanwhile, Figs. 9 to 11 depict some examples of molecules produced during the single-objective optimization for ESR1, ACAAI, and PHGDH. respectively.
19

---

346
0.047
0.048
5.01
0052 483 0,525
0.063 504 0.558
0.065 487 0.434
0.065 5.46 0.855
0.065 3:52 0.443
0.066 422 0,280
0066 369 0.598
0.069 5.18 0.33
0.073 4.8 0692
0.077 4.77 0.515
0.78 3.4 0.597
0.088 550 0.556
0.079 3.26 0.499
0.079 4.95 0531
0,081 3:34 0.507
Figure Molecules produced during the multi-objective optimization for ESRI. The legends denote K (nMM) L. SAJ and QED1.
0.181 4.66 0.503
0.24 4.88 0612
0.24 4.80 0.408
0.246 4.46 0.451
0.288 502 0.457
0.76 4.45 0.533
0,281 5.03 0580
0.301 4.25 0.566
0.306 464 0.900
0.327 3,84 0.455
0.333 4,02 0.449
0.344 499 0613
0.344 4.33 0.044
0.362 4.74 0.470
0.358 5:37 0.508
0.368 0.626
0.375
0.387 0.500
0.384 4.3
0,408 4.45 0.479
Figure 7: Molecules produced during the multi-objective optimization for ACAAI The legends denote Kp (nM) 4, SAL and QED
20

---

0.503
0.619
0.113 0468
0.113 0.505
0.490
0.156 0.421
0.408
0.172 3.57 0/66
0.181 4.22 0.423
0.181 0.458
0.184 0.486
0191 3.78 0416
0.201 4.08 0362
0.211 4.78 0.500
0.215 4.73 0.443
0.218 407 0.386
Figure 8: Molecules produced during the multi-objective optimization for PHGDH. The legends denote KD (nM) J. SA, and QED?.
ag
oa
003533
0.0377
0.0390
0.0411
0.0447
Figure Molecules produced during the single-objective optimization for ESRI. The legends denote KD(nM) J.
21

---

0.0729
0.0767
0.0793
0.0820
0.0834
0.0863
0.0863
0.0907
0.0923
0.3939
Figure 10: Molecules produced during the single-objective optimization for ACAAI. The legends denote Kp (nM) J.
0.0071
0.0073
0.0076
0.0080
0.2081
Figure 11: Molecules produced during the single-objective optimization for PHGDH. The legends denote (n)M)) J.
22

---

A.6.2 Structure-constrrined Optimization
1752 390 0.873
1502 310 0.873
3.577 330 0.733
2.347 291 0.507
2:57 3.10 0873
Figure 12: Molecules produced during the structure-constrranee optimization from CI to C2 for PHGDH. The legends denote (D(uM) SAT and QED1.
0.32 385 0.403
2,428 3.03 0.299
0.367 381 0.171
2554 337 0216
0186 3.50 0219
0287
0248
1278
1463 0.296
1215
Figure 13: Molecules produced during the structure-constraines optimization from C2 to C3 for PHGDH. The legends denote (n(uM) 4. SAL and QED1.
A.7 Broader Impact
We introduce novel generative model for jointly modeling molecule sequences and their target properties, which potentially leads to more capable and efficient molecule design algorithm. One potential negative impact could come from the misuse of the generative modeling in designing molecules or biological sequences for harmful purposes. Therefore, developing suitable safeguards and regulations will be crucial mitigate potential negative impacts.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Sec. 3 and Appendix.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See Sec. 6.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Justification: See Sec. 3.2.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See App. A.3.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
25

---

Answer: [Yes]
Justification:
Code is released our project page.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
wwwsssssssoolioiooeoooooooooo Ponl)cyl for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ /niips.co.o wwtpccccpppccceeeeccec.cccccccct wwwbbiccgguids..... /CooeSubmbia /tter//uuiideeoeeuuumuimiiiii ssinnnsiina ceiioniylcc for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See App. A.3 and Sec. 5.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: See Sec. 5 and App. A.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
26

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See App. A.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that NeurIPS Code of Ethics https: : /ttp///eerrp/.ee/ee//ee cc ct/pcc//pp/....ccc httpppbicccccchccccceeeeeee.eeee
didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the
Answer: [Yes]
Justification: None.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See App. A.7.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
27

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: None.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: None.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi wnwerswwthcodeee thcode. com/ datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
28

---

Answer: [No]
Justification: None.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [No]
Justification: None.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification: None.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "54uFuQ0vWn", "reviewer": "Reviewer_D3n3", "summary": "This work proposes a novel molecular optimization framework Latent Prompt Transformer (LPT)，modeling latent distribution, molecule sequences and properties distributions conditioned on the latent distribution. They uses MCMC in MLE training and conditional generation. Additionally, they propose an online learning algorithm to extrapolate to feasible regions supporting the desired property. The framework demonstrates promising results across various molecular design tasks.", "questions": "1. How is the transition from equation 4 to equation 5 derived? It would be beneficial to provide a detailed derivation for this step, as it is not immediately apparent.\n2. When formulating p(y∣z), how do you choose σ\\sigmaσ and how does it affect performance? Since properties can be uniquely determined by the molecule, i.e., p(y∣x) is deterministic, this suggests that in the probability model shown in Fig. 1 (Left), p(y∣z) should be very deterministic. However, this creates difficulties for sampling p_\\theta(z_0|y) in eq.7, because when \\sigma is small, \\nabla\\log p_\\theta(z_0|y) is very small for most z_0, leading to very slow MCMC convergence, especially with poor initialization.\n3. How robust is your model to the quality of the Oracle?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The online learning approach aligns well with real-world scenarios.\n2. The experiments are comprehensive, and the introduction of a new task for conditional generation and design trajectory comparison is expressive.\n3. The paper discusses and makes efforts to alleviate the bottleneck of computational efficiency. It also improves sample efficiency by reweighting, making the method more feasible.", "weaknesses": "- The clarity of the paper could be improved in several aspects:\n  1. Adding an illustration or diagram of the training and optimization process would make the methodology easier to understand.\n  2. Some formulas lack derivations and explanations, as raised in Question 1.\n- I am concerned that using MCMC sampling for training and inference may impact the model's practicality. Could you please compare the training and sampling speed with the baselines and discuss this issue in the limitations section?\n- Minor Issues:\n1. Typo error: \"2rd\" in Table 1.\n2. Using \"Kd\" values might lead readers to mistakenly think they are wet lab results, while in paper they represent docking scores.\n3. In Algorithm 2, 'step (c) Update LPT on synthetic dataset using **Algorithm 2** using MLE.' Should it be Algorithm 1 here?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review. We appreciate your recognition of our work's alignment with real-world scenarios, comprehensive experiments, and efficiency improvements. We'll address your comments and questions point by point.\n\n> W1. Clarity improvements:\n\nWe'll add a diagram illustrating the training and optimization process in the revised version and provide more detailed derivations (See Q1).\n> W2. MCMC sampling practicality:\n\nThank you for raising this important point. We hope to clarify that with careful design, MCMC is not our computational bottleneck. We'll include this discussion in the revised paper.\n\nIn offline pretraining, we sample $z \\sim p(z|x,y) \\propto p(z)p(x|z)p(y|z)$, where $z = U_\\alpha(z_0)$ with 15-step Langevin dynamics. In online learning or optimization, we use a persistent Markov chain that amortize the sampling across all optimization steps, reducing steps to 2. This speedup focuses on the optimization phase for real-world applicability.\n\nTo address your concern comprehensively, we add:\n\n1. **Comparison of model size and optimization time**: We compare optimization time after pretraining generative models on molecules. LPT shows comparable computation time to Bayesian optimization (BO) methods with much lower model size for LPT. This is for multiple-property objective (MPO) experiments in Practical Molecular Optimization (PMO) from Sec. 5.4.\n\n  | Method           | Assemb.     | Model      | Pretrain | Model Size (M)  | Time (min)   |\n  |------------------|-----------------|----------------|----------------|-----------------|-----------------|\n  | LSTM HC SMILES   | SMILES      |  RNN       |   Y      |  98.9       | 3           | \n  | VAE-BO SMILES    | SMILES      |  RNN & VAE       |   Y      |  17.9       | 17          |\n  | VAE-BO SELFIES   | SELFIES     |  RNN & VAE       |   Y      |  18.7       | 21          |\n  | LPT(Ours)              | SELFIES     |Transformer & MCMC |   Y      |  4.3        | 15          |\n\n2. **Breakdown of major computational costs for a single batch of 256 samples**:\n\n  | Computational Costs(s)   | Offline pretraining      | Online optimization     |\n  | --------------------    | --------------| -------------- |\n  | Posterior Sampling of $z$          |  5.147        |   0.294        |\n  | Molecule Generation     |  0.021        |   0.993        |\n  | Property Prediction     |  0.039        |   -        |\n\nThanks for the question. We shall consider improving the offline model learning speed in the future work and discuss this explicitly in the limiations.\n\n>W3. Minor issues:\n   - We'll correct the typo \"2rd\" to \"2nd\" in Table 1.\n   - Good point about \"Kd\" values. We'll clarify these are computational predictions, not wet lab results.\n   - You're correct, it should be Algorithm 1 in Algorithm 2 step \\(c\\). We'll fix this error.\n\n> Q1. Derivation from equation 4 to 5:\n\nWe'll provide a detailed derivation in the revision. Please refer to **Global Response Point 1**. \n\n\n>Q2. Formulating $p(y|z)$ and $\\sigma$ choice:\n\nThank you for this insightful question. You're correct that $p(y|x)$ is deterministic by nature. However, introducing a small error term $\\sigma$ in generative models is a common and necessary practice, aligning with various regression models and latent space optimization techniques.\n- In probabilistic regression models, including Gaussian Process Regression, $y = f(x) + \\epsilon$, where $\\epsilon \\sim N(0, \\sigma^2)$. Here, $\\sigma$ represents observation noise or model uncertainty as in Gómez-Bombarelli et al. (2018).\n- In Variational Autoencoders (VAE) for continuous variables, the property encoder typically outputs both a mean $\\mu$ and a standard deviation $\\sigma$ for each dimension, modeling $p(z|y)$ as $N(\\mu, \\sigma^2)$ such as Jiang et al.(NeurIPS 2020).\n- Latent Space Bayesian Optimization methods like LSBO (Tripp et al., NeurIPS 2020) use similar noise terms when fitting its Gaussian Process surrogate model.\n\nIn our case, modeling $p(y|z)$ with a small $\\sigma$ serves several purposes:\n- Account for potential inaccuracies in the oracle function or property prediction.\n- Provide smoothness to the optimization landscape, potentially aiding in convergence.\n- Allow for a degree of uncertainty in the latent space, which can be beneficial for exploration (in online learning LPT).\n\nYou raise an excellent point about the challenges in sampling $p_\\theta(z_0|y)$ when $\\sigma$ is small. One possible solution is to anneal $\\sigma$, by starting from a big value and gradually reducing it. \n\nReferences:\n\n[1] Gómez-Bombarelli et al. Automatic chemical design using a data-driven continuous representation of molecules. (ACS central science 2018)\n\n[2] Jiang et al. Multi-objective Deep Data Generation with Correlated Property Control. (NeurIPS 2020)\n\n[3] Tripp et al. \"Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining\" (NeurIPS 2020)\n\n>Q3. Model robustness to Oracle quality:\n\nWe've added experiments varying Oracle noise levels to assess robustness on single-objective QED optimization tasks under the budget of 25K. We use noised oracles as $y_{noise}=y_{true}+e$, where $e\\sim N(0, \\sigma^2)$, varying $\\sigma$ as a percentage of the property range. The results show our model is robust to the noised oracles.\n   \n| Oracle Noise  | 1st      | 2nd      |3rd      |Top-50   |\n| -------- | ---------| -------- |-------- |-------- |\n|  w/o     |  0.948   |   0.947  | 0.947    | 0.940±0.003   |\n|  1%      |  0.947   |   0.947  | 0.946    | 0.939±0.004   |\n|  5%      |  0.946   |   0.946  | 0.944    | 0.936±0.005   |\n|  10%     |  0.945   |   0.945  | 0.942    | 0.932±0.006   |\n\nThank you for these insightful comments. They will help improve the paper's clarity and completeness. We hope our responses have addressed your queries satisfactorily. If so, **we would appreciate your consideration in raising your rating of our submission**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "do9vSvBbBB", "reviewer": "Reviewer_jGf3", "summary": "The authors introduce a new conditional generative model for molecules. The model is called the Latent Prompt Transformer (LPT). A conditional model capable of generating new molecules with desired target properties is very useful in de-novo molecular design since we often want to design new molecules with some set of target properties. The authors initially train the LPT model on existing pairs of known molecules and property values. They then iteratively shift the distribution of the model towards regions with desired target properties. This results in a model capable of generating new molecules that are likely to have the desired target properties. In the experimental results section, the authors apply LPT to optimization tasks where the goal is to find new molecules that have high objective values, where the objective value here is some measurable desirable property of the molecule (i.e. the molecule’s binding affinity to some target of interest). Results show that the LPT model succeeds at these tasks, generating new molecules with higher binding affinity than baseline approaches. Additionally, the authors show that LPT can succeed at multi-objective optimization (generating new molecules with more than one desired target property). In particular, they show that LPT can generate new molecules that achieve high binding affinity, as well as high QED (Quantitative Estimate of Druglikeness) and minimal SA (synthetic accessibility) scores. In this case, results show that LPT is still able to generate molecules with higher binding affinity scores than baseline methods, while also having QED scores above 0.4 and SA scores below 5. Finally, the authors also show that LPT can be successfully applied to a new structure-constrained optimization task.", "questions": "See Weaknesses section above for suggestions/points of discussion.", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: LPT is novel as it is (as far as I am aware) the first autoregressive conditional generative model designed specifically for the task of conditional molecule generation. \nQuality: The paper is very well-written. Additionally, the figures and tables are all of good quality - they are both easy to parse and do a nice job of displaying results. Figures 1 and 2 do a very nice job of illustrating the author’s method. I especially like the use of Figure 2 to illustrate the distribution shift towards molecules with higher binding affinity. \nClarity: The paper is clear and easy to follow from start to finish. The figures and tables are clear and easy to read. The way the authors set up, trained, and applied the LPT model is clear. \nSignificance: De-novo molecular design is one of the most relevant/significant tasks in computational biology. In particular, finding molecules that bind to targets of interest is at the core of de-novo drug design. The paper shows that the author’s LPT model outperforms baseline approaches for relevant molecular design tasks such as finding new molecules with high binding affinity to targets of interest. This paper is therefore significant and of interest to the community.", "weaknesses": "Conditional generative models are not themselves novel as they have been successfully applied to a variety of text and image generation tasks. Much of the methodology in this paper involves taking existing methods in conditional generative modeling and applying them in a new domain: molecule generation. However, as far as I am aware this is the first time that one of these types of autoregressive conditional generative model has been designed for molecules, so this approach is novel from the perspective of methods for de-novo molecular design. I therefore think that this is a minor weakness and that this paper should be accepted.\n\nLatent-space Bayesian optimization (LS-BO) approaches are mentioned in related work as an alternative approach, but not directly compared to in experiments. I actually don’t think that this direct comparison is strictly necessary for the paper to be accepted because LS-BO is an orthogonal approach and the authors do compare LPT to a good number of state-of-the-art generative modeling approaches. However, a direct comparison showing that LPT performs as well as state-of-the-art LS-BO methods would strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and comprehensive review of our paper. We greatly appreciate your positive feedback on the originality, quality, clarity, and significance of our work. We're particularly grateful for your recognition of LPT's novelty and significance in de-novo molecule design.\n\nRegarding the weaknesses you identified:\n\n>W1. Novelty of conditional generative models:\n\nWe  appreciate your insightful observation about the novelty of our approach. Indeed, while conditional generative models have been applied in other domains, we believe our work makes a significant contribution by being the first to combine latent variable modeling with autoregressive molecule generation specifically for molecule design. This combination leverages the expressive power of Transformer-based autoregressive modeling for molecule generation and the efficiency of low-dimensional latent space sampling and design. This synergy is particularly advantageous in addressing the complexities of molecular structures and navigating the vast chemical space. Furthermore, it enhances our online learning algorithm, improving both design outcomes and sample efficiency. This integrated approach facilitates more effective exploration and optimization in molecular design.\n\n>W2. Comparison with Latent-space Bayesian Optimization (LS-BO):\n\nWe appreciate this suggestion and have added additional experiments comparing LPT with prominent BO-based optimization methods, which will be included in the revised version of our paper. We report the multi-property objectives (MPO) experiments under a limited oracle budget of 10K queries on the practical molecular optimization (PMO) benchmark [Gao et al., 2022]. We include the strongest BO-based baseline Gaussian process BO (GP-BO) [Tripp et al., 2021], LS-BO baselines including VAE-BO with both trained with SMILES [Gómez-Bombarelli et al., 2018] and SELFIES strings, and LSTM HC SMILES, which is the best generative molecule design method in PMO and is the same as reported in Table 5 of our main paper. Note that GP-BO optimizes the GP acquisition function with graph genetic algorithm in an inner loop.\n\n| Method           | Amlodipine      | Fexofenadine   | Osimertinib    | Perindopril     | Ranolazine      | Zaleplon        | Sum   |\n|------------------|-----------------|----------------|----------------|-----------------|-----------------|-----------------|-------|\n| LSTM HC SMILES   | 0.593±0.016     | **0.725±0.003**    | **0.796±0.002**    | 0.489±0.007     | 0.714±0.008     | 0.206±0.006     | 3.523 |\n| GP-BO              | 0.583±0.044     | 0.722±0.005    | 0.787±0.006    | 0.493±0.011     | **0.735±0.013**     | 0.221±0.072     | 3.541 |\n| VAE-BO SMILES              | 0.533±0.009     | 0.671±0.003    | 0.771±0.002    | 0.442±0.004     | 0.457±0.012     | 0.039±0.012     | 2.913 |\n| VAE-BO SELFIES              | 0.516±0.005     | 0.670±0.004    | 0.765±0.002    | 0.429±0.003     | 0.452±0.025     | 0.206±0.015     | 3.038 |\n| LPT              | **0.608±0.005**     | 0.714±0.003    | 0.784±0.011    | **0.511±0.002**     | 0.682±0.007     | **0.245±0.003**     | **3.544** |\n\nAs shown, LPT performs comparably to the strong GP-BO baseline and LSTM HC SMILES, while significantly outperforming LS-BO (VAE based) methods across multiple tasks. LPT achieves the highest overall sum score, demonstrating its robust performance across diverse molecular optimization objectives. Since BO-based baselines often require careful parameter tuning, we directly report the numbers from the PMO benchmark that is carefully tuned. In future work, we plan to conduct more thorough comparisons in binding affinity experiments to further validate LPT's performance against these baselines.\n\nReferences:\n\n[1] W. Gao et al. Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization. NeurIPS, 2022.\n\n[2] A. Tripp et al. A fresh look at de novo molecular design benchmarks. NeurIPS 2021 AI for Science Workshop, 2021.\n\n[3] R. Gómez-Bombarelli et al. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 2018.\n\nWe believe these additions further solidify our work's contribution to de-novo molecular design. Thank you again for your valuable feedback and recommendation for acceptance. We look forward to the opportunity to present a strengthened version of our work that incorporates your valuable suggestions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C8TNKELMxC", "reviewer": "Reviewer_rWJi", "summary": "The paper introduces an approach for molecule design, by leveraging recent advancements in conditional generative models for language and image generation. It contains three steps: (1) learnable prior distribution, (2) molecule generation model and (3) property prediction model. The experiments are comprehensive and convincing to me.", "questions": "1. At line 29, I don't think existing methods decouple the training of the generative model from the property conditioned optimization, such as [1] and [2].\n2. What's the difference between the proposed method and variational autoencoder (VAE)? I see both methods assume the prior on the latents, and there are works that generated molecules from VAE such as [1] and [2].\n3. I noticed that the paper proposes Langevin dynamics to get z, but why not directly use an encoder to encode z from data like VAE does? MCMC-based method seems more computationally heavy to me.\n4. At line 154, I'm still confused about how to sample z | y using Langevin dynamics. Especially how can we compute the gradient of the p(z_0|y)?\n\n[1] Multi-objective Deep Data Generation with Correlated Property Control. NeurIPS 2022.\n[2] Property controllable variational autoencoder via invertible mutual dependence. ICLR 2020.", "limitations": "Yes. Limitations are well discussed.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The task that the proposed method tries to solve is of great importance in a lot of real-world applications such drug design and protein design.\n2. The formulation of multi-objective optimization task is quite novel and well defined.\n3. The experiments are comprehensive and convincing to me.", "weaknesses": "1. Motivation to use the current framework is unclear to me. Why we use Langevin dynamics to sample latent variables rather than directly using the VAE framework to introduce the prior?\n2. I think the dependence between molecules/properties cannot be secured as there is no reconstruction loss on molecules. As you may see in Eq. (5) and the equation under Eq. (6), the information when predicting y from z cannot be back-propagated to x. Then how to align x and y?\n3. Minor: \n(a) line 42: would suggest to write down the full name of MCMC before using its abbreviation for general audience.\n(b) typo at equation (1): subscript of o^p_m(x).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed and constructive review. We appreciate the time and effort you’ve put into evaluating our work. We will address your comments and questions point by point:\n\n>W1: Motivation for framework:\n\nThank you for your question. We kindly refer you to the **Global Response Point 1** for a detailed explanation of our motivation.\n\n>W2: Dependence between molecules/properties:\n\nOur approach ensures the dependence between molecules and properties through:\n\n1.**Joint probability modeling**: We learn $p(x,y,z) = p(z)p(x|z)p(y|z)$, where $z = U_\\alpha(z_0)$ and $z_0 \\sim \\mathcal{N}(0, I_d)$, tying molecules and properties through the shared latent space $z$.\n\n2.**MLE learning of joint model** (Equation 5, Algorithm 1):\na) Sample $z$ from $p(z|x,y)$ using Langevin dynamics:\n$z_{\\tau+1} = z_\\tau + s\\nabla_z \\log p(z_\\tau|x,y) + \\sqrt{2s}\\epsilon_\\tau$\nThe gradient for sampling $z$ is (as in line 137):\n$\\nabla_{z_0} \\log p(z_0|x,y) = \\nabla_{z_0} \\log p_0(z_0) + \\nabla_{z_0} \\sum_{t=1}^T \\log p(x^{(t)}|x^{(<t)},z) + \\nabla_{z_0} \\log p(y|z)$\nThis ensures $z$ is sampled from regions consistent with both the observed molecule structure and properties, while the noise term $\\epsilon_\\tau$ enables exploration of the latent space.\nb) Update model parameters to maximize likelihood given sampled $z$ above.\n\n>W3: Minor issues:\n\nWe’ll spell out MCMC on first use and correct the typo in Equation 1.\n\n>Q1: Decoupling of generative model and property optimization:\n\nThank you for this important point. Our original statement was too broad. In our introduction, we primarily focused on the online molecule design. We will revise our statement to reflect this and makes sure to cite the paper you mentioned:\n\n\"While some approaches to molecule design have treated generative modeling and property optimization separately, recent work such as [1] and [2] has shown promising integration of these aspects. Our approach builds upon these ideas, extending them in the context of online molecule design.\"\n\n>Q2: Comparsion of VAE and MCMC\n\nIn the offline setting (Sec. 3.2), both MCMC-based and VAE-based algorithms are valid for approximating MLE learning. However, VAE-based methods with autoregressive decoding can suffer from posterior collapse without careful design. We conducted additional experiments on single-objective QED optimization with 25K molecule-property pairs in offline settings. We parameterized the encoder as Transformer models with the same layers but with full attention rather than causal attention. Performance degradation for our model was observed:\n\n| Model  | 1st      | 2nd      |3rd      |Top-50   |\n| -------- | ---------| -------- |-------- |-------- |\n|  LPT    |  0.948   |   0.947  | 0.947    | 0.940±0.003   |\n|  LPT-VAE     |  0.944   |   0.944  | 0.943    | 0.928±0.008   |\n\nIn the online setting (Sec. 3.4), which is the main focus of our paper, MCMC-based LPT is more compatible with our needs. It can adapt to distribution shifts with a learned prior model and flexibly control exploration and exploitation to improve sample efficiency. MCMC-based optimization should be compared to other online optimization techniques such as Bayesian optimization (BO) with a pre-trained VAE. Our detailed comparisons on the PMO benchmark show that our methods outperform VAE-based methods (Table is shown in **Global Response Point 3**), even when strong BO techniques are used.\n\n\n>Q3: Computational efficiency of MCMC based methods.\n\nWe clarify that with careful design, MCMC is not our computational bottleneck. We’ll include this discussion in the revised paper.\n\nIn offline pretraining, we sample $z \\sim p(z|x,y) \\propto p(z)p(x|z)p(y|z)$, where $z = U_\\alpha(z_0)$ with 15-step Langevin dynamics. In online learning, we use a persistent Markov chain that amortizes sampling across optimization steps, reducing steps to 2, thus enhancing real-world applicability.\n\nTo address your concern comprehensively, we compare our method with strong VAE-based online optimization methods in PMO benchmark:\n\n**Comparison of model size and optimization time**: We compare optimization time after pretraining generative models on molecules with baselines in PMO bechmark with multiple-property objective (MPO) experiments. LPT shows comparable computation time to VAE-based Bayesion Optimization(BO) method.\n\n| Method           | Assemb.     | Model      | Pretrain | Model Size (M)  | Time (min)   |\n|------------------|-----------------|----------------|----------------|-----------------|-----------------|\n| VAE-BO SMILES    | SMILES      |  RNN & VAE       |   Y      |  17.9       | 17          |\n| VAE-BO SELFIES   | SELFIES     |  RNN & VAE       |   Y      |  18.7       | 21          |\n| LPT(Ours)              | SELFIES     |Transformer & MCMC |   Y      |  4.3        | 15          |\n\n>Q4: Sampling $p(z_0|y)$ using Langevin dynamics:\n\nThis process is explained in Equation 7 of the main paper. The posterior distribution is given by: $z_0 \\sim p_\\theta(z_0|y) \\propto p_0(z_0)p_\\gamma(y|z = U_\\alpha(z_0))$. To sample from this distribution using Langevin dynamics, we iterate:\n$z_0^{\\tau+1} = z_0^\\tau + s\\nabla_{z_0} \\log p_\\theta(z_0|y) + \\sqrt{2s}\\epsilon^\\tau$ where:\n$\\nabla_{z_0} \\log p_\\theta(z_0|y) = \\nabla_{z_0} \\log p_0(z_0) + \\nabla_{z_0} \\log p_\\gamma(y|z = U_\\alpha(z_0))$\n\nHere, $p_0$ is a standard Gaussian distribution, so its gradient is straightforward to compute. The second term is also computable. For example, if we assume $p_\\gamma(y|z)=\\mathcal{N}(s_\\gamma(z),\\sigma^2)$ as in line 121, the second term becomes $1/\\sigma^2(y-s_\\gamma(z=U_\\alpha(z_0)))\\nabla_{z_0}s_\\gamma(z=U_\\alpha(z_0))$ using the chain rule. This can be implemented by pytorch autograd.\n\nThank you for your valuable feedback on our submission. If you have any additional concerns, please let us know. We hope our responses have addressed your queries satisfactorily. If so, **we would appreciate your consideration in raising your rating of our submission**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Yc2pQp3NJG", "reviewer": "Reviewer_vrpK", "summary": "This paper presents LPT, a novel transformer model for conditional molecule sequence design and generation. LPT first generates latent vectors from a learnable prior distribution, then autoregressively generates molecule sequence taking the latent vector as prompt. Comprehensive experiments show that LPT achieves state-of-the-art performance on a variety of molecule, protein and DNA design benchmarks.", "questions": "See Weaknesses part.", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Propose a novel transformer model LPT  and a novel framework to achieve property conditioned generation of molecule sequence.\n- Strong and solid experiment results on multiple benchmark datasets.\n- Good, clear and well-organized writing.", "weaknesses": "- As SMILES strings of molecules have grammars so not every SMILES string can be decoded to a real molecule. Authors are encouraged to report validity rate of the generated SMILES strings.  \n- What is the dimension/size of the latents in experiments? How will the dimension/size of the latents impact the performance? Authors are suggested to give a discussion or conduct ablation study experiments about this problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your thorough review and positive assessment of our paper. We appreciate your recognition of our novel model LPT, the framework for property-conditioned molecule sequence generation, and the strong experimental results across multiple benchmarks. We are pleased that you found our writing clear and well-organized.\nWe would like to address the weaknesses and questions you raised:\n\n> W1: Validity rate of generated SMILES strings:\n\nThank you for this important suggestion. In our molecule design experiments, we used the SELFIES representation (Krenn et al., 2020) instead of SMILES, as briefly mentioned in Section A.1.1. SELFIES is a 100% robust molecular string representation that ensures all generated strings correspond to valid molecules. As a result, we achieved 100% validity in all our molecule design experiments. We acknowledge that we should have made this point more explicit in the main text, and we will clearly state this in the revised paper.\n\nWe agree that the validity checking of SMILES string generation is a good metric for model sanity check. Although not reported in the current version of the paper, we did perform this sanity check by evaluating SMILES generation with our model pretrained on ZINC. We achieved a validity rate of 0.999 for 10k generated molecules, which  validates our model's capability to generate valid molecules and the model can potentially capture the chemical rules. We will include this result in the revised paper.\n\n> W2: Dimension/size of latents and its impact on performance:\n\nThank you for highlighting this important aspect. We conducted additional ablation studies on the impact of latent variable dimensionality in the challenging single-objective PHGDH experiment with the same number of oracle calls (50k):\n| Latent Variables  | 1st      | 2nd      |3rd      |Optimization Time (Hours)   |\n| -------- | ---------| -------- |-------- |-------- |\n|  2       |  0.10    |   0.14   | 0.21    | 8       |\n|  4       |  0.08    |   0.10   | 0.11    | 12      |\n|  8       |  0.07    |   0.09   | 0.12    | 15      |\n\nWe observe a trade-off between performance and computational time. Increasing the number of latent variables improves results but increases optimization time. The performance gain diminishes from 4 to 8 variables, while computational cost continues to rise.\nBased on these findings, we chose 4 latent vectors for z (each size 256, total dimension 1024) in our main experiments, balancing performance and computational efficiency for practical applications. We shall include this discussion in our revised paper.\n\n\n\nOnce again, we thank you for your constructive feedback. We are committed to improving our paper based on your suggestions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

Thank you for your insightful and constructive comments on our submission. We have added derivation of Eqn. 5 for completeness and clarified our motivation for using MCMC-based methods in online molecule design.

1. **Derivation of Equation 5**

$$
\begin{align*} 
   \nabla_\theta \log p_\theta(x, y) 
   &= \frac{\nabla_\theta p_\theta(x, y)}{p_\theta(x, y)} \\\\
   &= \frac{1}{p_\theta(x, y)} \int \nabla_\theta p_\theta(x, y, z_0)  dz_0 \\\\
   &= \int \frac{p_\theta(x, y, z_0)}{p_\theta(x, y)} \nabla_\theta \log p_\theta(x, y, z_0) dz_0 \\\\
   &= \int p_\theta(z_0 | x, y) \nabla_\theta \log p_\theta(x, y, z_0) dz_0 \\\\
   &= \mathbb{E_{p_\theta(z_0|x, y)}} \left[ \nabla_\theta \log p_\theta(x, y, z_0) \right] \\\\
   &= \mathbb{E_{p_\theta(z_0|x, y)}} \left[ \nabla_\theta \log p_\beta(x|U_\alpha(z_0)) + \nabla_\theta \log p_\gamma(y|U_\alpha(z_0)) + \nabla_\theta \log p_0(z_0) \right] \\\\
   &= \mathbb{E_{p_\theta(z_0|x, y)}} \left[ \nabla_\theta \log p_\beta(x|U_\alpha(z_0)) + \nabla_\theta \log p_\gamma(y|U_\alpha(z_0)) \right].
\end{align*}
$$




2. **Motivation of MCMC-based learning and optimization**
Our choice of Langevin dynamics for online molecule design was driven by several key advantages:

- **Avoiding posterior collapse**: Langevin dynamics circumvents the issue of posterior collapse, a significant challenge in training VAEs with autoregressive decoders for sequential data like molecules. In VAEs, the strong autoregressive decoder can often reconstruct the data by relying solely on the one-step-ahead ground truth, ignoring the latent codes entirely. This leads to a trivial local optimum where the posterior collapses to the prior, carrying no useful information [1]. In contrast, our method samples from the posterior distribution of z, iteratively refining the latent prompts and increasing their likelihood given the observed data. This iterative refinement ensures that the latent space remains informative throughout training, avoiding the collapse problem inherent in VAEs [2].

   [1] Fu, Hao, et al. “Cyclical annealing schedule: A simple approach to mitigating kl vanishing.” NAACL (2019).

   [2] Pang, Bo, et al. “Generative text modeling through short run inference.” EACL (2021).
- **Adaptability to distribution shifting with a learnable prior**: Our method incorporates a learnable prior, crucial for adapting to scenarios where desired property values lie outside the initially learned distribution. This adaptability is essential in practical online molecule design, where we aim to generate molecules with properties not present in the initial training data. Unlike VAEs with fixed priors, our learnable prior adjusts to new target regions in the property space with a small number of online data samples. This is quantitatively demonstrated in our ablation study (Appendix A1.2, Table 7), which shows significant performance degradation when using a fixed Gaussian prior.
- **Flexible exploration-exploitation trade-off**: Our use of Langevin dynamics for property-conditioned generation allows nuanced control over the exploration-exploitation trade-off during test-time. As shown in Equation 8 and the ablation study in Table 8, we can adjust the guidance weight in the sampling process to balance between exploring new molecular structures and exploiting known high-performing regions of the latent space. This flexibility is crucial for improving sample efficiency in practical molecule design. There are existing works that leverage pretrained VAEs and other optimization techniques (such as Bayesian Optimization) to adapt VAEs in online scenarios.
- **Unfied framework**:  MCMC sampling allows us to develop both offline and online learning algorithms within a unified framework as LPT.
- **Handling multi-modal posteriors**: The posterior distribution of latent variables given a desired property may be multi-modal. Langevin dynamics is better equipped to handle such multi-modal distributions compared to the typically unimodal approximate posteriors used in VAEs.

3. **Comaprsion with VAE-based optimization methods.**
We provide additional comparsion with VAE-based methods on the practical molecular optimization(PMO), where multi-property objectives are optimized under a limited budget of 10K. The numbers are taken from PMO benchmark.

| Method           | Amlodipine      | Fexofenadine   | Osimertinib    | Perindopril     | Ranolazine      | Zaleplon        | Sum   |
|------------------|-----------------|----------------|----------------|-----------------|-----------------|-----------------|-------|
| VAE-BO SMILES              | 0.533±0.009     | 0.671±0.003    | 0.771±0.002    | 0.442±0.004     | 0.457±0.012     | 0.039±0.012     | 2.913 |
| JT-VAE BO Fragments              | 0.519±0.009     | 0.667±0.003    | 0.775±0.002    | 0.430±0.004     | 0.508±0.012     | 0.046±0.012     | 2.945 |
| VAE-BO SELFIES              | 0.516±0.005     | 0.670±0.004    | 0.765±0.002    | 0.429±0.003     | 0.452±0.025     | 0.206±0.015     | 3.038 |
| LPT SELFIES             | **0.608±0.005**     | **0.714±0.003**    | **0.784±0.011**    | **0.511±0.002**     | **0.682±0.007**     | **0.245±0.003**     | **3.544** |

Meta-review: The reviewers have appreciated the novelty of the approach, although some of it stemming from the fact that the approach was applied to molecules for the first time; they also liked the fact that the experimental work was comprehensive. The reviewers had concerns about the computational weight of the MCMC sampling, but those were alleviated already by the proposed approach and additional comparisons to SOTAs were performed during the rebuttal period to show practical efficiency of the method. The authors contributed a number of additional results including comparisons to additional methods during the rebuttal. Overall the paper is a solid contribution and of interest to the NeurIPS community.
Decision: Accept (spotlight)