MF-LAL: Drug Compound Generation Using Multi-Fidelity Latent Space Active Learning

## Abstract

Abstract Current generative models for drug discovery primarily use molecular docking as an oracle to guide the generation of active compounds. However, such models are often not useful in practice because even compounds with high docking scores do not consistently show experimental activity. More accurate methods for activity prediction exist, such as molecular dynamics based binding free energy calculations, but they are too computationally expensive to use in a generative model. To address this challenge, we propose Multi-Fidelity Latent space Active Learning (MF-LAL), a generative modeling framework that integrates a set of oracles with varying cost-accuracy tradeoffs. Unlike previous approaches that separately learn the surrogate model and generative model, MF-LAL combines the generative and multi-fidelity surrogate models into a single framework, allowing for more accurate activity prediction and higher quality samples. We train MF-LAL with a novel active learning algorithm to further reduce computational cost. Our experiments on two disease-relevant proteins show that MF-LAL produces compounds with significantly better binding free energy scores than other single and multi-fidelity approaches.

## 1 Introduction

Generative models for de novo drug design have gained significant interest in machine learning for their promised ability to quickly generate new compounds for specific applications. However, generating compounds with real-world biological activity remains a fundamental challenge (Handa et al., ; Coley et al., ). One of the main difficulties is the computational evaluation of compound-protein binding affinities. The generated compounds are often highly novel, so an activity predictor trained with existing experimental data is insufficient due to poor out-of-distribution generalization (Chatterjee et al., ; Ji et al., ). Instead, physics-based methods that model 3D interactions between compound and target are commonly used.

Due to its speed, molecular docking is the prevalent physics-based method to evaluate novel compounds by generative models (Eckmann et al., ; Jeon & Kim, ; Lee et al., ; Noh et al., ; Fu et al., ; Peng et al., ; Guan et al., ; ). However, docking is a relatively poor predictor of activity (Pinzi & Rastelli, ; Handa et al., ; Coley et al., ; Feng et al., ), so it would be desirable to apply more accurate binding free energy calculation techniques (Pinzi & Rastelli, ; Feng et al., ). These techniques, based on molecular dynamics simulations, are currently considered the most reliable approach to predict affinity (Moore et al., ; Cournia et al., ). However, they have not been used by generative models due to their high computational cost (Thomas et al., ), with a single compound-protein pair taking hours to days to simulate on a powerful computer (Wan et al., ). Thus, neither docking nor binding free energy techniques alone can guide the real-world application of generative models.

Multi-fidelity surrogate models aim to fuse multiple data sources as oracles spanning a range of accuracy and cost (Fernández-Godino et al., ).They are frequently learned using an active learning approach, where a model selects or generates queries that it is most uncertain about to send to a chosen oracle (Ren et al., ). The results from the oracle are then added to the training data of the model. We will focus on “query synthesis” approaches (Angluin, ), where the model generates its own queries to send to the oracles, speeding up learning compared to approaches that query oracles with samples from a fixed candidate set.

Combining docking (low fidelity) and binding free energy (high fidelity) using multi-fidelity surrogate models holds promise to make generative models more practical. Yet, the use of multi-fidelity methods in drug discovery has been limited. Prior work from Hernandez-Garcia et al. () uses a generative model to generate query compounds with high acquisition function values computed by a separate multi-fidelity surrogate model. However, since we want to generate query compounds to send to oracles at multiple fidelity levels, the distribution of optimal query compounds may differ across fidelities. A separate generative model is not aware of such differences across fidelity levels, hence it cannot send queries to the multi-fidelity oracles efficiently.

We aim to address the problem of multi-fidelity generation with Multi-Fidelity Latent space Active Learning (MF-LAL), an integrated framework for compound generation using multi-fidelity active learning. Instead of separating the generative model and surrogate model, we perform surrogate modeling and generation together at each fidelity level using a sequence of hierarchical latent spaces. This improves the quality of generated queries because there is a separate latent space and decoder specialized for each fidelity, and improves surrogate modeling and inter-fidelity information passing because each latent space can be organized for predicting at just that level. We use both docking and binding free energy methods as oracles in our multi-fidelity environment to achieve a favorable trade-off between cost and accuracy. In summary,

- •
we introduce a novel multi-fidelity surrogate and generative modeling framework, MF-LAL, which integrates data from multiple fidelity levels to generate high-quality samples at the highest fidelity (binding free energy).
- •
we employ an active learning approach with a novel query generation technique that ensures compounds generated at higher fidelities also scored well at lower fidelities, improving the quality of generated samples.
- •
we evaluate MF-LAL and baseline methods on a real-world problem setting involving optimizing the binding free energy of compounds against two disease-relevant human proteins, and find that MF-LAL generates compounds with significantly better scores than baselines.

## 2 Related Work

### 2.1 Molecular generative models

Generative models in drug discovery have gained much interest for their ability to quickly generate compounds with desired properties (Paul et al., ). Early works (Jin et al., ; Gómez-Bombarelli et al., ; You et al., ) focus on properties such as the octanol-water partition coefficient (logP) or quantitative estimate of drug-likeness (QED), which are of very limited practical utility (Coley et al., ; Xie et al., ). More recently, there has been an understanding that the binding affinity to a targeted protein is much more relevant for practical drug discovery (Xie et al., ; Eckmann et al., ; Fu et al., ).

One approach to guide generative models in optimizing compound binding affinity is to use an oracle for compound evaluation. This oracle can be applied to reinforcement learning (Jeon & Kim, ; Fu et al., ; Mazuz et al., ), VAEs (Eckmann et al., ; Noh et al., ), genetic algorithms (Spiegel & Durrant, ; Fu et al., ), diffusion models (Lee et al., ; Hoogeboom et al., ; Wu et al., ), or other generative frameworks Zhu et al. (). All of them use docking software, such as AutoDock (Morris et al., ), as the oracle, because it is the only reasonably fast option. However, docking is known to be inaccurate (Pinzi & Rastelli, ), and compounds with high docking scores do not consistently show experimental activity (Handa et al., ; Coley et al., ; Feng et al., ).

Molecular dynamics-based binding free energy calculations are much more accurate than docking (Moore et al., ; Cournia et al., ), but have not yet been applied to de novo generative drug design due to their high computational cost (Thomas et al., ). While Ghanakota et al. () use binding free energy calculations in combination with a molecular generative model, they focus on the optimization of an existing known lead compound. This allows them to rely on much cheaper relative binding free energy calculations, as opposed to the absolute binding free energy (ABFE) calculations needed for de novo design (Cournia et al., ).

Structure-based generative models are trained on 3D structures of protein-ligand pairs, and aim to predict a 3D ligand that fits in a given protein pocket with high binding affinity. Techniques include autoregressive generation (Peng et al., ) and diffusion modeling (Guan et al., ; ). Despite not needing an oracle like docking during the generation process, the generated compounds are still evaluated with docking as a post-processing step. This means structure-based generative models do not avoid the issue of inaccurate binding affinity prediction.

### 2.2 Multi-fidelity surrogate modeling

Multi-fidelity modeling methods aim to fuse multiple data sources of variable accuracy and cost (Fernández-Godino et al., ), and are widely used in scientific fields for surrogate modeling and uncertainty quantification (Brevault et al., ). A popular choice of surrogate model is a Gaussian process (GP), which performs well in low data settings and produces well-calibrated uncertainty estimates (Brevault et al., ). One such technique to apply GPs to multi-fidelity modeling is described by Wu et al. (), where a downsampling kernel is used to output predictions at each fidelity level. Other multi-fidelity surrogate modeling approaches utilize neural processes (Wang & Lin, ; Wu et al., ; ; Niu et al., ) and ordinary differential equations (Li et al., ) as an alternative to GPs.

Multi-fidelity models are frequently trained in an active learning fashion, where one uses an estimate of a model’s uncertainty to most efficiently acquire more datapoints from an oracle (Ren et al., ). In the multi-fidelity setting, this means iteratively querying across both the sampling space and each different fidelity oracle (Li et al., ; Hernandez-Garcia et al., ). Traditional active learning involves selecting from a fixed candidate set with the highest acquisition function value to query oracles with, which limits the training set to only existing samples. It also limits how much the model can learn with each query, since the maximally informative sample may not be present in the candidate set. Query synthesis approaches (Angluin, ) have been proposed to avoid this problem by using a generative model to generate new queries. Hernandez-Garcia et al. () have applied these ideas to drug discovery problems by training a generative model to optimize the acquisition function computed by a separate multi-fidelity surrogate model, which does not take into account the different distributions of optimal query compounds at each fidelity.

## 3 MF-LAL

Figure: Figure 1: Overview of Multi-Fidelity Latent space Active Learning (MF-LAL).
Refer to caption: x1.png

We introduce Multi-Fidelity Latent space Active Learning (MF-LAL), an integrated framework for compound generation using multi-fidelity Bayesian active learning. An overview of our framework is shown in Figure 1. We encode molecules into a hierarchy of latent spaces (left panel), one for each fidelity, and learn surrogates that predict the oracle output based on the latent vectors (middle panel). These oracles are used to reverse optimize in the latent spaces to generate new compounds with high predicted scores. The generated molecules are fed to an oracle at a chosen fidelity in an active learning loop, the output of which is used to re-train the latent representation and surrogate models (right panel). After training, we use the surrogates to reverse optimize in the highest fidelity latent space and generate compounds with high property scores at the highest fidelity. See Appendix A for details of our model and a diagram of the network architecture.

### 3.1 Learning multi-fidelity latent representations

#### Problem setup.

A multi-fidelity environment consists of a set of oracles $\{f_{1},\dots,f_{k},\dots,f_{K}\}$ that predict a property of interest, where the accuracy and cost of the predictions increase with the fidelity level $k$. We have a multi-fidelity dataset $\mathcal{D}$ consisting of $K$ distinct sets of molecules, one for each fidelity, $\mathcal{D}=\{\{x_{1}^{(i)},y_{1}^{(i)}\}_{i=1}^{N_{1}},\dots,\{x_{K}^{(i)},y_
{K}^{(i)}\}_{i=1}^{N_{K}}\}$. Each $y_{k}^{(i)}=f_{k}(x_{k}^{(i)})$ is the result from querying oracle $k$ with molecule $x_{k}^{(i)}$. The molecules are drawn from unknown distributions $p^{*}_{1},\dots,p^{*}_{K}$. We aim to approximate these distributions using generative models $p_{\theta_{1}},\dots,p_{\theta_{K}}$ with parameters $\theta_{1},\dots,\theta_{K}$. Note that $p_{1}^{*}\neq\dots\neq p_{K}^{*}$, meaning we must learn separate generative models for each fidelity level, as opposed to previous approaches that learn a single generative model for all fidelities.

#### Latent representation.

To learn the generative models, we first learn an encoding of the input molecule to the lowest fidelity latent space. Specifically, we use a single probabilistic encoder $q_{\phi}$ parameterized by $\phi$ that encodes a molecule $x$ into mean and variance vectors $\mu_{1}$ and $\sigma_{1}$. The latent vector $\mathbf{z}_{1}\sim\mathcal{N}(\mu_{1},\sigma_{1})$, corresponding to the first (lowest) fidelity, is sampled from the resulting distribution. Since we want a separate latent space at each fidelity level, we define a set of probabilistic networks $h_{\xi_{1}}(\mathbf{z}_{1}),\dots,h_{\xi_{K-1}}(\mathbf{z}_{K-1})$ with parameters $\xi_{1},\dots,\xi_{K-1}$ that pass information between latent spaces. Specifically, $h_{\xi_{k}}$ takes the vector $\mathbf{z}_{k}$ as input and outputs a mean and variance vector in the subsequent latent space, $\mu_{k+1},\sigma_{k+1}$. We sample from this distribution to obtain latent vector $\mathbf{z}_{k+1}$, i.e. $\mathbf{z}_{k+1}\sim\mathcal{N}(\mu_{k+1},\sigma_{k+1})$. We also define a set of probabilistic decoder networks $p_{\theta_{1}}(\cdot|\mathbf{z}_{1}),\dots,p_{\theta_{K}}(\cdot|\mathbf{z}_{K})$ to reconstruct the original molecule $x$ from the latent vectors. The use of a specialized decoder for each fidelity level improves reconstruction quality compared to previous methods that only use one, thus making the generated samples more tailored for their fidelity level.

We represent molecules using SELFIES strings (Krenn et al., ). The encoder and decoder of MF-LAL are fully-connected neural networks that use a flattened, one-hot encoded SELFIES string. See Table 2 for comparison with other encoder and decoder designs.

#### Surrogate modeling.

In order to generate molecules with high property scores, we aim to learn differentiable surrogates $\hat{f}_{1},\dots,\hat{f}_{K}$ that approximate the oracles and use them for reverse optimization in the latent spaces. Each surrogate $\hat{f}_{k}$ maps from its corresponding latent vector $\mathbf{z}_{k}$ to an estimate of $f_{k}$. We use gradient-based optimization to find a point in a given latent space that has a high property score predicted by the surrogate, which can then be decoded to a molecule (Gómez-Bombarelli et al., ). The $h_{\xi_{k}}$ networks, which pass information between latent spaces, allow us to re-use information learned about the molecule’s binding properties at lower fidelities to aid in prediction at the higher fidelities without having to re-learn it using large amounts of high-fidelity data. This is because training the surrogate models organize each latent space (Tevosyan et al., ) for property prediction at that level, and so the latent vectors contain information about the binding properties useful for predicting the oracle output that can then be passed to higher fidelities. Additionally, the use of separate latent spaces for each fidelity level, as opposed to previous approaches that use only a single latent space shared across all levels, improves surrogate modeling performance because each latent space can be organized for prediction at just that level.

We use stochastic variational Gaussian process models (SVGPs, (Hensman et al., )) with parameters $\lambda_{1},\dots,\lambda_{K}$ for the surrogates. We chose SVGPs for their speed of training and ability to train with minibatches. Specifically, $\hat{f}_{k}$ is given by $\hat{f}_{k}\sim\mathcal{GP}(m_{\lambda_{k}}(x),\Sigma_{\lambda_{k}}(x,x^{
\prime}))$ where $m(x)$ and $\Sigma(x,x^{\prime})$ are the mean and covariance kernels of the SVGP. To train the model, we jointly minimize the evidence lower bound (ELBO) (Kingma & Welling, ) of the latent encodings and marginal log likelihood (MLL) of the GP models. Given a training molecule $x$ and associated oracle output $y$ at fidelity $k$, we minimize

$$ $\begin{split}L(\phi,\xi_{k-1},\theta_{k},\lambda_{k};k,x,y)&=\mathbb{E}_{ \mathbf{z}_{k}\sim g(\cdot|x)}\log\frac{p_{\theta_{k}}(x|\mathbf{z}_{k})}{g( \mathbf{z}_{k}|x)}+\int p(y|\hat{f}_{k}(x))p(\hat{f}_{k}(x)|x)d\hat{f}_{k},\\ &\text{where }g(\mathbf{z}_{k}|x)=\begin{cases}q_{\phi}(\mathbf{z}_{k}|x)& \text{if }k=1\\ h_{\xi_{k-1}}(\mathbf{z}_{k-1})&\text{else}\\ \end{cases}\\ \end{split}$ (1) $$

where the first term is the ELBO and the second is the MLL of the GP. While the loss is only evaluated at fidelity $k$, it is backpropagated through to all lower fidelities. Additionally, in our implementation, we approximate the MLL GP loss using the ELBO (Hensman et al., ) for improved scalability.

### 3.2 Bayesian active learning for sample-efficient training

Figure: Algorithm 1 Active learning for MF-LAL

Training a multi-fidelity surrogate model requires significant computational resources, especially to gather data at the highest fidelity level. Instead of passively collecting training data, we develop a Bayesian active learning approach to efficiently query the oracles, allowing us to make fewer queries to the most expensive oracles. As show in Algorithm 1, our active learning cycle consists of first generating a molecule to query at a chosen fidelity, querying the oracle to obtain the property score, appending the result to the dataset, and then retraining the model. We repeat the process until some computational budget is reached.

Similar to Kandasamy et al. (), we start with only querying the oracle at the lowest fidelity level $k=1$ and increase to higher fidelities when the model’s uncertainty falls below certain thresholds. We use the posterior variance of the GP surrogate $\Sigma_{\lambda_{k}}$ to measure the model’s uncertainty. Specifically, during active learning, we repeatedly generate a latent vector $\mathbf{z}_{k}$ at fidelity $k$ that decodes to a query compound. If $\Sigma_{\lambda_{k}}(\mathbf{z}_{k})<\gamma_{k}$, where $\gamma_{k}$ is the uncertainty threshold, then we permanently increment $k$ by one for all subsequent queries. Otherwise, $k$ remains the same. Once at the highest fidelity, we keep running active learning until some computational budget is reached. We use this stepwise approach to ensure all surrogates have enough training data to make accurate predictions in high property areas of the latent spaces, thus leading to high property compounds being generated.

We generate query molecules from the latent space using the upper confidence bound (Auer, ) as the acquisition function. The method we use to generate molecules is described later in Section 3.3. To ensure that generated compounds remain similar to the training set of drug-like molecules, we also add an L2 regularization term on the latent vector. The acquisition function is thus given by

$$ $a(\mathbf{z}^{(i)}_{k},k)=m_{\lambda_{k}}(\mathbf{z}^{(i)}_{k})+\beta\Sigma_{ \lambda_{k}}(\mathbf{z}^{(i)}_{k})-||\mathbf{z}^{(i)}_{k}||^{2}_{2}$ (2) $$

where $\mathbf{z}^{(i)}_{k}$ is a point in latent space $k$ and $\beta$ is an exploration hyperparameter. $m$ and $\Sigma$ are the mean and covariance kernels of the GP surrogate. We set $\beta=1$ during active learning, and $\beta=0$ after during inference to only focus on the most promising compounds.

### 3.3 Generating molecules with high property scores

Figure: Algorithm 2 MF-LAL molecule generation procedure

Our goal is to generate compounds at fidelity $k$ that maximize some generation objective. To accomplish this, we perform gradient-based optimization to find a point $\mathbf{z}_{k}$ in the $k$th latent space that maximizes the generation objective, and then decode $\mathbf{z}_{k}$ into a molecule using $p_{\theta_{k}}$. For our generation objective, we do not want to simply maximize $\hat{f}_{k}$, but instead the upper confidence bound (Eq. 2) to ensure exploration during active learning. In addition, we also introduce a novel likelihood-based term to the generation objective that encourages the model to only sample compounds at higher fidelities that also scored well at the lower fidelities. Specifically, when generating a molecule at fidelity $k$, we maximize the likelihood that the molecule would also be generated at fidelity $k-1$ with a high property score. This additional term greatly restricts the area of the chemical space explored by the high fidelity oracles, reducing the computational cost wasted on non-promising areas and making the use of high-cost oracles feasible. It also means the higher fidelity latent spaces encode a more limited distribution of compounds, improving the quality of samples generated from those latent spaces. Indeed, we show that the likelihood term is critical for strong performance (Table 2).

To compute the likelihood of a point $\mathbf{z}_{k}$ at fidelity $k$, we first generate a set of $M$ high-scoring compounds at fidelity $k-1$. Next, we map those points to a sum of Gaussians in the $k$th latent space using $h_{\xi_{k-1}}$, giving us a set of $M$ parameters $\{(\mu^{(j)}_{k},\sigma^{(j)}_{k})\}_{j=1}^{M}$. We then measure the likelihood of point $\mathbf{z}_{k}$ in the generated sum of Gaussians distribution. This guarantees that the compounds generated in latent space $k$ are also likely to have been generated in $k-1$ with high scores. Thus, we effectively reduce the size of the chemical space that must be explored at fidelity $k$ to only compounds that have already shown promise at the lower fidelities. Mathematically, maximizing the likelihood of a point $\mathbf{z}^{(i)}_{k}$ in latent space $k$ is equivalent to maximizing the probability density function evaluated at that point:

$$ $P\left(\mathbf{z}^{(i)}_{k}|\{(\mu^{(j)}_{k},\sigma^{(j)}_{k})\}_{j=1}^{M} \right)=\sum_{j=1}^{M}\frac{1}{\sqrt{2\pi(\sigma^{(j)}_{k})^{2}}}\exp\left(- \frac{(\mathbf{z}^{(i)}_{k}-\mu^{(j)}_{k})^{2}}{2(\sigma^{(j)}_{k})^{2}}\right)$ (3) $$

The full generation algorithm is detailed in Algorithm 2. In our implementation, we vectorize optimization across all $M$ latent space points simultaneously. In order to encourage diversity in generated compounds, we also add a term to the generation objective that measures the average pairwise cosine similarity between the $M$ points, and aim to minimize it:

$$ $\frac{1}{M^{2}}\sum_{i=1}^{M}\sum_{j=1}^{M}s(\mathbf{z}^{(i)}_{k},\mathbf{z}^{ (j)}_{k})$ (4) $$

where $s(A,B)$ is the cosine similarity between vectors $A$ and $B$.

## 4 Experiments

### 4.1 Experimental setup

We define a multi-fidelity environment for binding affinity which uses four oracles, each of which takes a molecule as input and outputs an estimate of its binding affinity with increasing accuracy:

- 1.
Linear regression ($f_{1}$). Simple linear regression model trained on experimental data from the BRD4(2)/c-MET target from BindingDB (Liu et al., ) to predict the $K_{i}$, a measure of binding affinity. Morgan fingerprints are used to represent the molecule.
- 2.
AutoDock4 ($f_{2}$) (Morris et al., ). Uses 3D geometric and charge information from the protein and compound to estimate the binding energy.
- 3.
Ensembled AutoDock4 ($f_{3}$) (Morris et al., ). Same as above, except we dock the compound into the binding pockets of eight BRD4/five c-MET cocrystal structures that were solved with different known ligands, and then take the minimum predicted energy. This ensemble approach is generally more accurate than using a single protein structure (Amaro et al., ).
- 4.
Absolute binding free energy (ABFE) ($f_{4}$) (Heinzelmann & Gilson, ). A binding free energy method applicable to de novo drug discovery that uses molecular dynamics simulations to accurately predict the binding energy.

We target the BRD4(2) and c-MET proteins (PDB 5UF0 and 5EOB), both of which are implicated in human cancer development, although through different biological mechanisms. We chose these targets because ABFE is already well-validated on them and known to have good agreement with experimental data (Heinzelmann & Gilson, ). See Appendix B for further experimental details and analysis of the oracles, including experiments confirming that our higher fidelity oracles are more costly yet more accurate at distinguishing experimental actives from inactives.

Each model is provided with an initial dataset of random ZINC250k (Irwin et al., ) compounds queried at each fidelity (see Appendix B for further details). To compare models, we run each in an active learning loop using a fixed computational budget of 7 days, and then generate 15 unique compounds at the highest fidelity predicted to have the best scores. We then run these compounds through ABFE and compare their scores. See Appendix C for additional results showing the oracle-predicted binding energy of the generated query compounds over the active learning process.

### 4.2 Baselines

We compare MF-LAL with the following baselines:

- •
SF-VAE (only ABFE / only docking) (Gómez-Bombarelli et al., ). Uses a simple single-fidelity GP as a surrogate model that is used to guide optimization in the latent space of a vanilla VAE, representing a simple single-fidelity approach. This consists of two separate baselines, one where the GP is trained only on ABFE data and one where it is trained on only docking data.
- •
REINVENT (only ABFE / only docking) (Olivecrona et al., ). An RL-based molecular generation technique which we use to optimize ABFE, and separately, docking score. Represents a simple single-fidelity approach using a modern generative model.
- •
VAE + 4x SF-GP. Uses a vanilla VAE model except with four independent GP surrogates, one for each fidelity, all using the single latent space as input. To be contrasted with MF-LAL, which uses multiple connected latent spaces instead of a single one.
- •
VAE + MF-GP. Similar to above, except using a single multi-fidelity GP model (Wu et al., ) instead of four independent single-fidelity GP models.
- •
MF-AL-GFN (Hernandez-Garcia et al., ) GFlowNet generative model used to optimize the predicted score from a multi-fidelity GP model. This baseline represents the state of the art in multi-fidelity generation, where generative and surrogate models are separated.
- •
MF-AL-PPO (Hernandez-Garcia et al., ). Same as above except using the PPO RL algorithm instead of a GFlowNet as the generative model.
- •
Pocket2Mol (Peng et al., ). 3D structure-based drug design model that takes a protein pocket as input and outputs a 3D molecule via diffusion. Unlike the other methods, does not use any binding affinity oracle during generation.

The first two baselines are single-fidelity methods, where we use both only ABFE and only docking as the single fidelity. The next four baselines, as well as MF-LAL, are multi-fidelity. Pocket2Mol does not utilize any oracle during generation. Evaluation of generated compounds from all baselines is done with ABFE.

### 4.3 Results

**Table 1: Evaluation of generated compounds at highest fidelity. The mean and top 3 ABFE values are shown for 15 compounds sampled from each method after active learning for 7 days.**
| Method | BRD4(2) ABFE (kcal/mol) | c-MET ABFE (kcal/mol) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Mean $\pm$ s.d. | 1st | 2nd | 3rd | Mean $\pm$ s.d. | 1st | 2nd | 3rd |
| SF-VAE (only ABFE) | -0.9 $\pm$ 2.7 | -5.7 | -2.9 | -2.9 | -1.2 $\pm$ 3.0 | -4.4 | -3.9 | -3.1 |
| SF-VAE (only docking) | -3.1 $\pm$ 2.8 | -6.1 | -5.3 | -4.8 | -2.8 $\pm$ 3.4 | -5.9 | -5.8 | -5.1 |
| REINVENT (only ABFE) | -3.9 $\pm$ 3.4 | -8.7 | -8.3 | -8.2 | -2.9 $\pm$ 3.7 | -6.5 | -5.8 | -5.1 |
| REINVENT (only docking) | -3.1 $\pm$ 4.9 | -11.0 | -6.2 | -5.7 | -2.6 $\pm$ 5.0 | -8.0 | -6.8 | -5.9 |
| VAE + 4x SF-GP | -2.3 $\pm$ 3.1 | -8.0 | -5.5 | -5.3 | -1.8 $\pm$ 2.5 | -6.3 | -5.9 | -5.1 |
| VAE + MF-GP | -1.3 $\pm$ 3.3 | -4.9 | -3.1 | -2.0 | -3.3 $\pm$ 2.9 | -9.7 | -7.7 | -4.2 |
| MF-AL-GFN | -2.5 $\pm$ 2.2 | -6.5 | -5.8 | -5.1 | -3.1 $\pm$ 1.8 | -5.5 | -4.5 | -4.1 |
| MF-AL-PPO | -2.8 $\pm$ 2.5 | -9.2 | -6.5 | -5.2 | -4.0 $\pm$ 2.4 | -7.0 | -6.6 | -5.8 |
| Pocket2Mol | -4.8 $\pm$ 4.6 | -9.8 | -9.8 | -9.0 | -2.2 $\pm$ 4.2 | -4.5 | -3.9 | -3.2 |
| MF-LAL (ours) | -6.3 $\pm$ 4.0 | -12.0 | -11.3 | -10.2 | -6.5 $\pm$ 3.2 | -12.9 | -7.7 | -6.5 |

Figure: Figure 2: Visualization of MF-LAL generated molecules. The top 3 molecules and associated ABFE scores are shown for both the BRD4(2) (top row) and c-MET (bottom row) targets.
Refer to caption: x2.png

Table 1 reports the average and top 3 ABFE scores of 15 compounds generated by MF-LAL, as well as those generated by the baseline methods, following active learning for 7 days. We ran each method separately for two targets, BRD4(2) and c-MET. We filtered generated compounds such that all had QED (Bickerton et al., ) $>0.4$, SAscore (Ertl & Schuffenhauer, ) $<4$, and no rings with $\geq 7$ atoms. We also only allowed compounds that fit these criteria to be queried during active learning. We filtered compounds that did not meet the criteria after generation, instead of performing multi-objective optimization, because most generated compounds from single-objective optimization already had a QED/SAscore in the range of typical drug compounds (Bickerton et al., ; Ertl & Schuffenhauer, ).

We find that the average ABFE scores of the compounds generated by MF-LAL, as well as those of the top three compounds, are significantly better (lower kcal/mol) than the corresponding scores of compounds generated by the baseline methods for both targets. The difference in average ABFE score (predicted binding free energy) between MF-LAL and the top baseline is -1.5 kcal/mol for BRD4(2) and -2.5 kcal/mol for c-MET, which are significant margins. Thus, our method outperforms both single and multi-fidelity techniques, as well as a 3D structure-based method (Pocket2Mol, Peng et al. ()) that does not use any binding affinity oracle. This suggests MF-LAL is most capable at generating compounds with real-world activity, since binding free energies computed with ABFE are the gold standard prediction method. Note, too, that the multi-fidelity techniques other than MF-LAL performed mostly similarly to the single-fidelity methods. This suggests that successfully taking advantage of multiple fidelities requires an architecture which, like that used in MF-LAL, is tailored to generating compounds for multiple fidelity oracles.

The molecular structures of the top 3 compounds generated by MF-LAL for both targets are shown in Figure 2. The compounds are structurally diverse, and none of them have close similars in the training set or in large datasets like PubChem (Kim et al., ). This shows the ability of MF-LAL to generate promising and novel structures. There is a small shared scaffold across 2 of the generated c-MET compounds consisting of a pyridazine, but this is not necessarily undesirable as it shows the model has found a high-property region of chemical space. The scaffold may be a promising starting point for development of empirical structure-activity relationships and lead optimization by medicinal chemists. Additionally, the pairwise Tanimoto similarity among the 15 compounds generated by MF-LAL is less than 0.2 for both targets, further indicating that our method generates a structurally diverse set.

**Table 2: Ablations. The mean and top 3 ABFE-computed energies are shown for 15 compounds sampled from each method after active learning for 7 days.**
| Method | BRD4(2) ABFE (kcal/mol) | c-MET ABFE (kcal/mol) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Mean $\pm$ s.d. | 1st | 2nd | 3rd | Mean $\pm$ s.d. | 1st | 2nd | 3rd |
| MF-LAL | -6.3 $\pm$ 4.0 | -12.0 | -11.3 | -10.2 | -6.5 $\pm$ 3.2 | -12.9 | -7.7 | -6.5 |
| -fid. 1 | -6.1 $\pm$ 0.7 | -7.7 | -7.6 | -7.4 | -6.0 $\pm$ 1.1 | -8.8 | -7.0 | -6.0 |
| -fid. 2 | -5.1 $\pm$ 2.0 | -8.5 | -6.5 | -6.0 | -5.2 $\pm$ 2.5 | -8.0 | -7.3 | -6.1 |
| -fid. 3 | -4.2 $\pm$ 3.1 | -9.2 | -5.9 | -5.7 | -4.2 $\pm$ 3.5 | -9.8 | -7.1 | -6.1 |
| -fid. 4 | -2.4 $\pm$ 3.2 | -8.6 | -4.3 | -3.4 | -3.1 $\pm$ 3.0 | -7.6 | -6.7 | -5.1 |
| w/o likelihood term | -3.6 $\pm$ 4.4 | -11.9 | -10.7 | -10.0 | -3.8 $\pm$ 3.7 | -10.9 | -7.7 | -6.3 |
| Transformer enc/dec | -6.2 $\pm$ 4.0 | -11.5 | -11.3 | -9.8 | -6.5 $\pm$ 2.9 | -11.6 | -7.6 | -6.5 |
| GCN enc/dec | -5.9 $\pm$ 3.0 | -10.9 | -10.1 | -9.3 | -6.1 $\pm$ 4.6 | -11.1 | -7.5 | -6.5 |

Table 2 reports results from various ablations of the MF-LAL architecture. We experimented with removing each fidelity level individually, removing the likelihood term from the generation objective, and replacing the fully-connected encoder/decoder networks with a Transformer (Vaswani et al., ) and graph convolutional network (GCN, Kipf & Welling ()). The results show that all fidelities contribute to some degree to performance, although $f_{1}$ (the linear regression model) seems to add very little. Removing the likelihood term greatly reduced the performance of our method, showing that the approach of only querying compounds at higher fidelities that also scored well at lower fidelities is critical for strong performance. Replacing the fully-connected encoder/decoder with a Transformer had little effect on performance, so we used the simpler fully-connected version. Finally, changing the molecular representation to a graph and replacing the fully-connected encoder/decoder with GCNs (Kipf & Welling, ) resulted in slightly worse performance.

## 5 Discussion and Conclusion

We present Multi-Fidelity Latent space Active Learning (MF-LAL), an integrated framework for generative and multi-fidelity surrogate modeling. Our experiments show that MF-LAL generates compounds with significantly higher activity, as predicted by a gold-standard binding free energy oracle, than other single and multi-fidelity approaches. Thus, MF-LAL shows promise as a way to generate compounds with real-world binding while incurring a reasonable computational cost.

Limitations of our approach include a limited set of oracles and a potential lack of synthesizability of the generated compounds, since SAscore is known to be imperfect (Skoraczyński et al., ). Future work could include using a reaction-aware generative model that generates more synthesizable molecules (Horwood & Noutahi, ).

## 6 Acknowledgments

This work was supported in part by Army-ECASE award W911NF-07-R-0003-03, the U.S. Department Of Energy, Office of Science, IARPA HAYSTAC Program, CDC-RFA-FT-23-0069, NSF Grants #2205093, #2146343, and #2134274. MKG has an equity interest in and is a cofounder and scientific advisor of VeraChem LLC.

## Appendix A Model details

The encoder, decoder, and $h$ networks are all 3-layer feed-forward networks with ReLU activations and a 512-dimensional hidden layer. Each latent space has 64 dimensions. Molecules are represented using SELFIES strings (Krenn et al., ), and fed to the network using a flattened one-hot encoding representation. The loss on the latent encodings is the ELBO, which is the sum of the KL divergence and the cross-entropy loss of the reconstruction.

The surrogate models consist of a 4-layer deep kernel (Wilson et al., ) to encode the input and a Matern kernel for the covariance function. To accelerate training, we use an approximate GP trained with the ELBO loss (Hensman et al., ).

At each active learning step, we train the whole model from scratch until convergence with the Adam optimizer using a learning rate of 0.0001. For the molecule generation procedure using gradient-based optimization, we use the Adam optimizer with a learning rate of 0.1 for 100 epochs.

### A.1 MF-LAL model diagram

Figure: Figure 3: Diagram of MF-LAL architecture. An input molecule is encoded into the first latent space $\mathbf{z}_{1}$ using a neural network. The networks $h_{\xi_{1}},\dots,h_{\xi_{K-1}}$ transform points in $z_{1}$ to the higher fidelity latent spaces. Each latent space has an associated decoder, which reconstructs the original molecule, and a GP surrogate model for that fidelity level.
Refer to caption: x3.png

## Appendix B Experimental details

All experiments were conducted on a server with 8 RTX 2080 Ti GPUs. For our model and each baseline, we performed a random hyperparameter search with 20 trials using only the first 3 fidelities, and took the set of hyperparameters with the best generated samples at $f_{3}$ following 3 hours of active learning. We excluded ABFE from the hyperparameter search due to computational cost, and just used the same set of hyperparameters for all subsequent experiments using all 4 fidelities.

Each model was provided with an initial dataset of random ZINC250k (Irwin et al., ) compounds evaluated at each fidelity. Each fidelity had 5 random compounds selected, except for the first fidelity, where we supplied 200,000 compounds and associated oracle outputs. This was because we wanted a large dataset of compounds to train the encoder and decoder at the first fidelity level, ensuring that generated compounds were reasonable, and running $f_{1}$ was almost instantaneous. For the more expensive oracles, however, we let active learning generate compounds to query to most efficiently use computational resources.

### B.1 Oracles

For all oracles, we estimated the cost (for the baselines that require it) using the average run time over 10 samples with random input compounds. We also computed the ROC-AUC (shown below) of each oracle for the BRD4(2) target to confirm that the higher cost oracles are more accurate. To do this, we generated a dataset of known active and presumed inactive compounds for BRD4(2), and then ran each oracle on all of the compounds. The actives were retrieved from the BRD4(2) target from BindingDB with $K_{i}<10\mu M$, and the inactives were generated using DUD-E (Mysinger et al., ). As expected, the ROC-AUC increases with higher fidelity, as well as the computational cost.

#### Linear regression (cost: 0.1s, ROC-AUC: 0.59)

We used BRD4(2) and, separately, c-MET data from BindingDB (Liu et al., ) to train a simple linear regression model. The input to the model was 2048-bit Morgan fingerprints, and the output was the experimental binding energy in $K_{i}$, converted to kcal/mol.

#### AutoDock4 (cost: 4s, ROC-AUC: 0.65)

We prepared the AutoDock4 grid files using AutoDockTools (Morris et al., ). Arbitrary ligands were prepared using obabel (O’Boyle et al., ) with pH 7.4 and gasteiger partial charges. We used AutoDock-GPU (Santos-Martins et al., ), a GPU-accelerated version of AutoDock4, for all computation. For each ligand, we performed 20 random restarts and selected the minimum predicted energy.

#### Ensembled AutoDock4 (cost: 44s, ROC-AUC: 0.70)

Same as above, except we used the minimum energy from 8 separate AutoDock4 runs using the same ligand and each of the following protein crystal structures (listed as PDB IDs): 5ues, 5uet, 5uev, 5uez, 5uf0, 5uvs, 5uvy, 5uvz for BRD4(2), and 2wd1, 4deg, 4dei, 4r1v, 5eob for c-MET

#### Absolute binding free energy (ABFE) (cost: 9:20hrs, ROC-AUC: 0.92)

We use the Binding Affinity Tool (BAT.py) implementation (Heinzelmann & Gilson, ) for absolute binding free energy calculation, available at [https://github.com/GHeinzelmann/BAT.py](https://github.com/GHeinzelmann/BAT.py). For BRD4(2), we use the short tevb calculations, which were introduced recently to reduce computational cost (Heinzelmann et al., ). For c-MET, we use the standard SDR method, but we found we could reduce simulation times for all components to 30% of their original amounts and still retain strong performance. All molecular dynamics simulators are run with AMBER with GPU support. As BAT.py requires a starting pose for the ligand, we used the pose generated from AutoDock4. We additionally wrote custom scripts to parallelize molecular dynamics runs across all available GPUs.

### B.2 Baselines

The details of each baseline are as follows:

- •
SF-VAE (only ABFE / only docking) (Gómez-Bombarelli et al., ). The VAE encoder and decoder, and GP surrogate, are set up similarly to those in MF-LAL. The upper confidence bound is used as an acquisition function.
- •
REINVENT (only ABFE / only docking) (Olivecrona et al., ). Used the code available at [https://github.com/MarcusOlivecrona/REINVENT](https://github.com/MarcusOlivecrona/REINVENT).
- •
VAE + 4x SF-GP. The VAE encoder and decoder, and GP surrogates, are set up similarly to those in MF-LAL. We also use the same acquisition function and generative procedure as MF-LAL for this baseline, except without the need to map points between latent spaces.
- •
VAE + MF-GP. We use the Multi-Fidelity Max Value Entropy acquisition function for selecting compounds during active learning (Takeno et al., ), and a linear truncated fidelity kernel (Gardner et al., ) for the GP surrogate.
- •
MF-AL-GFN (Hernandez-Garcia et al., ) Used the code available at [https://github.com/nikita-0209/mf-al-gfn](https://github.com/nikita-0209/mf-al-gfn).
- •
MF-AL-PPO (Hernandez-Garcia et al., ). Provided in the same codebase as the one referenced above.
- •
Pocket2Mol (Peng et al., ). Used the code available at [https://github.com/pengxingang/Pocket2Mol](https://github.com/pengxingang/Pocket2Mol).

## Appendix C Additional results

Figure: Figure 4: Oracle outputs during active learning. The y-axis shows the oracle-predicted binding energy of the generated query compounds, and the x-axis shows active learning progress.
Refer to caption: x4.png

Figure 4 shows the oracle-predicted binding energy of the compounds generated during the MF-LAL active learning process. We only show fidelities 2 and 3, because fidelity 1 is already supplied with a large initial dataset so there is little improvement in the query quality during active learning, and fidelity 4 does not show any noticeable improvement due to a relatively small number of queries made. For fidelities 2 and 3, we observe a marked improvement of the predicted binding energy over active learning, showing that MF-LAL successfully learns what makes a compound favorable at each fidelity level.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: I encourage the authors to address the following:

1. Perform docking and ABFE to known binder compounds.
2. Revisit the docking results taking into account the aforementioned issues.
soundness: 3
strengths: This work addresses one of the most important problems in de novo drug design, the translational efficiency that oracles transmit to the generation process. 

The hierarchical latent space, each corresponding to a fidelity level, is a clever design choice. This allows the model to specialize the latent representation for each fidelity, improving both surrogate modeling and the quality of generated compounds.

The use of SELFIES is really appropriate for this setup as this work is not focus on broader chemical exploration.

The introduction of a likelihood-based term in the generation objective is a smart technical trick. Specifically, when generating a molecule at fidelity level \(k\), the model maximizes the likelihood that this molecule would also be generated at fidelity level \(k-1\) with a high property score.

Surrogate models \(\hat{f}_k\) are directly trained on their corresponding latent representations \(z_k\), leveraging SVGPs. This integration ensures that the latent spaces are organized to optimize property prediction at each fidelity, enhancing the overall performance of both the generative and surrogate models. The use of SVGPs for surrogate modeling at each fidelity level introduces mathematical robustness. The joint minimization of the ELBO and the marginal log likelihood ensures that the latent representations are well-structured for property prediction
summary: The paper introduces a novel framework called Multi-Fidelity Latent Space Active Learning (MF-LAL) for de novo drug design. The primary challenge addressed is the generation of drug compounds with real-world biological activity, a task complicated by the limitations of current evaluation methods. Generative models often rely on molecular docking as an oracle to guide compound generation, but docking scores do not consistently correlate with experimental activity. Authors state that more accurate methods like molecular dynamics-based binding free energy calculations exist but are computationally prohibitive for large-scale generative tasks.
MF-LAL proposes integrating multiple oracles of varying fidelity level, each with different accuracy and computational cost, into a single generative modeling framework. Unlike previous approaches that separately train the surrogate model and the generative model, MF-LAL combines them, allowing for more accurate activity predictions and higher-quality sample generation. The framework employs a novel active learning algorithm to efficiently select which oracle to query at each step, further reducing computational overhead. Experimental results on two disease-relevant proteins demonstrate that MF-LAL outperforms both single and multi-fidelity baseline approaches in producing compounds with better binding free energy scores.
weaknesses: The whole technical approach is fantastic. However, this work has some issues conceptually and with the evaluation of its results.

While it's true that the degree of resolution of free binding energies estimation can be higher for some systems, this is taken as a pillar fact in this work while this might not be true. It is of utmost importance to check any physics-based protocols with compounds with known affinity and actually corroborate that the low to high-fidelity methods stand as such.

To that end, authors select 15 compounds for each method and compute ABFE metrics on all of them. However, they base a great deal of their analyses on 3 top scoring compounds per system. When looking at the mean of the 15 compounds generated vs each of the values for the top 3 compounds it's clear to see that they are outliers, which might not be bad per see (it is exactly what a generative campaign is looking for) but must be always followed by a careful analysis of the motifs the molecules generated. Scoring functions of any type, docking or ABFE, are biased towards certain types of chemical structures and assign a high score as a result of highly reactive topologies. For instance, best molecule in BRD4 has an SO2 group, known to be pan-reactive, best molecule in c-MET has an aldehyde group, also known to bind to many chemistries potently. The other two compounds in c-MET have nitro groups, with charges known to be rewarded by scoring functions. While those are valid molecule they would be potentially discarded since they would be considered artifacts.

Also, the approach is lacking from benchmarking the systems on docking and ABFE with compounds known to bind to them. That would give a more solid comparison point for generated molecules.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: 1. Maximizing the likelihood that the molecule would also be generated at fidelity k−1 with
a high property score apparently limits exploration space, and the oracle at the lowest level is likely to misguide the searching spot due to its poor accuracy.  This might be the reason for the high variance (especially a large gap between mean and top 3 values) in the predicted properties of 15 generated molecules. Moreover, as shown in Table 2, the top 3 results w/o likelihood term are not significantly lower than with it. The authors need to elaborate on the potential effect of using the likelihood constraint (eq 3) on constraining the exploration.
2. Using only 15 generated compounds is too few to evaluate statistically meaningful performance, which casts doubt on the conclusions. In particular, the top 3 values strongly rely on sampling. The authors may evaluate with more samples or discuss the limitations.
3. Training the proposed model is terminated with a total time limit, which causes an unsystematic convergence criterion.  As a result, the binding affinities of the top 2 and 3 of C-MET are much weaker than the top 1, whereas the top 3 values of BRD4 are in a similar range. The authors need to justify the consequences of their termination criterion (7 days) on the model performance. 
4. The authors define a set of probabilistic decoder networks that translate a latent vector at each fidelity level to a single original molecule. This reviewer wonders about the performance of each decoder in terms of success rates in reconstructing the original molecule. Do they all give the same correct molecule or show different success rates depending on the fidelity level?
soundness: 2
strengths: 1. Latent space active learning combined with generative AI to efficiently query the oracles
2. Multi-fidelity level latent space and decoder at each, leadting to higher accuracy
3. Well compromising between speed and accuracy by reusing the latent information from lower levels on higher levels. 
4. The use of ABFE for prediction is challenging in conventional approches, but this work has used it for evaluation thanks to the relatively high efficiency of the proposed method.
summary: The authors proposed a new approach to designing molecules binding to a given target protein. Conventional approaches are mostly based on docking methods which are known to be insufficient for reliable predictions of binding between molecules and a target protein. Absolute biniding free energies (ABFE) are the most accurate prection method for bining affinites, but they are too expensive to apply to many molecules. The authors resolved this problem  by using a novel multi-level fidelity latent space active learning (ML-LAL). The result shows that the proposed method outperformed baseline models. The idea of ML-LAL is new but the benchmark results need to be further justified because of the use of only 15 generated molecules in the evaluation.
weaknesses: 1. Only 15 generated compounds have been tested for evaluations, which seems too few to convince conclusions.
2. The mean and top 3 values of the 15 generated molecules have large differences,
3. Training at the highest fidelity level is terminated not by the training error threshold but the total computational time. Thus, no systematic convergence for a given target can be made.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: How is the reconstruction accuracy for the 4 seperate encoder-decoder networks?
The authors wrote in [Line 183-185] that "the use of a specialized decoder for each fidelity level improves reconstruction quality compared to previous methods, that only use one, thus making the generated samples more tailored for their fidelity level".

In the supplementary code base in the main.py, I saw that the weighting coefficient for the KL-prior is set quite low with ~0.08, indicating that the latent space is not really smooth and especially for the first fidelity level $z_1 \sim N(z_1 | \mu_1(x), \sigma_1(x))$ where 200,000 samples are available for training the VAE-1, "holes" in the latent space might appear. Could the author provide reconstruction learning curves over epochs? As the fidelity-level gets higher, the sparser the latent spaces $Z_k, k\geq 2$ become, since first these spaces are trained with less data. Although the authors provide the log-likelihood term of a mixture of Gaussians from previous latents, it is not clear how often a optimized latent $z_k$ can be decoded back to a valid SELFIES string.
soundness: 3
strengths: The paper is well written and motivated by the current methods in the literature which mostly rely on a single oracle function such as docking, without considering higher fidelity and reliable evaluation metrics. Since the usage of higher-fidelity oracles such as computing the absolute binding free energy (ABFE) is costly, approximating those computation with neural networks  as surrogate models is reasonable and brings the advantage that these surrogate models can be used for gradient-based optimization. The active learning component, usually applied in supervised learning, for querying samples from a given dataset with a costly oracle function, is here used for generation purposes where the optimization for promising ligands is done in a hierarchy of latent spaces using gradient-based methods. The authors show that the proposed loss terms for active learning, i.e., the acqusition function and likelihood terms from lower-fidelity levels are necessary for improved evaluation results, which is confirmed in their ablation studies.
summary: The authors introduces an active learning generative learning framework to design novel small molecules (ligands) that are optimized to have high binding affinity to a protein target using a set of oracle functions. Their method uses an increasing hierarchy of oracles, whose output becomes more reliable with higher-fidelity oracles with the downside that the oracle's execution become more computationally expensive.  
The authors propose to use an increasing set (which state the higher fidelity-spaces) of encoder-decoder architecture to embed a ligand from SELFIES-representation to a latent space, and decode the latent representation back for generation purpose. Crucially, the latent representation from the encoder(s), are suitable to train surrogate models for the multiple fidelities with the advantage that gradient-based optimization can be also performed in the latent spaces.  
The authors show that their proposed method which performs active learning on hierarchical latent spaces outperforms current string-based and 3d-based generative models in their experiments on two protein target that are involved in human cancer development.
weaknesses: The comparison to the 3d generative model Pocket2Mol is not fair since no optimization is done to sample ligands that have high binding affinity for that method. The (autoregressive) Pocket2Mol model also does not fall into the state-of-the-art for 3d generative anymore, although this method is faster in generation than current diffusion-based models.
I would be interested how the 1d/SELFIES-based MF-LAL framework compares against newer 3d generative models such as DecompDiff [1] or PILOT [2] without applying guidance or calling the oracles, since their methods might not be directly applicable within the multi-fidelity hierarchy.

[1] DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Design, https://proceedings.mlr.press/v202/guan23a.html.   
[2] PILOT: equivariant diffusion for pocket-conditioned de novo ligand generation with multi-objective guidance via importance sampling, https://pubs.rsc.org/en/content/articlelanding/2024/sc/d4sc03523b

---

confidence: 4
contribution: 1
initial_rating: 3
presentation: 3
questions: - Can you describe the method's sensitivity to hyperparameters?
- Can you please explain why one would want molecules which score well at high fidelities *and* low fidelities, instead of only caring about high fidelities?
- Can you comment on the statistical significance of your results?
soundness: 2
strengths: The problem studied by the authors is sensible and important to the field. The method proposed by the authors is, broadly speaking, reasonable (although I describe some weaknesses below). I think the paper is well-written and the presentation is good.
summary: This paper proposes a multi-fidelity active learning method to produce molecules with desirable properties, given multiple different oracles with varying costs. The method uses an encoder to embed molecules into a latent space, with stochastic variational Gaussian process (SVGP) models trained for each fidelity. The authors perform experiments on multiple protein targets.
weaknesses: I have two main criticisms of the method itself:

1. A key motivation for the method is to  "ensures compounds generated at higher fidelities also scored well at lower fidelities, improving the quality of generated samples." (lines 075-077). This hypothesis seems questionable: why would one want this? The fundamental challenge in multi-fidelity AL is that the observations between different fidelities are not very well-correlated. Is it not possible that most interesting molecules at high-fidelity score poorly at lower fidelities?
2. The method uses SVGPs in quite a few places, which raises concerns about robustness. SVGPs are very sensitive to the location of their inducing points, and tend to overestimate variance away from their inducing points (see <https://arxiv.org/abs/1606.04820>). This means:
  - Line 7 in algorithm 1 may not be implemented accurately if the variance is overestimated
  - The distribution of the initial dataset will likely determine the location of the inducing points, and with the UCB objective it seems plausible that nothing away from the initial distribution will therefore be chosen. At the very least, the behavior will depend enormously on the locations of the inducing points.

Aside from this, I thought the experiments had some significant correctness issues. First and foremost, the results in Table 1 are described as "significant" despite a very small sample size of 15 and considerable variance in the scores of the generated molecules. I think the authors need to apply a statistical test to their results so that the differences between methods can be evaluated more objectively. I understand that the expensive nature of the computations requires a small sample size, but this is _exactly_ the scenario where statistical tests are helpful.

Second, I think the authors are missing a "standard MF-AL" method as a baseline- the most "standard" baselines use VAE-GP models which may not fit the data well. I recommend either an autoregressive GP (ie prior mean for fidelity $k$ is the posterior mean for fidelity $k-1$) or a "co-kriging" model where a single GP has a kernel between inputs and fidelities. EI or UCB could be used as an acquisition function, and the kernel could either be a Matern 5/2 or Tanimoto kernel between molecule fingerprints.

Finally, more broadly the sensitivity of the method to hyperparameters is not discussed or explored. My guess is that the performance is extremely sensitive to parameters like UCB $\beta$ or the locations of the inducing points, and this may prevent the method from being applied successfully in other problems. It would be good to see this discussed more.

---

confidence: 5
contribution: 1
initial_rating: 5
presentation: 3
questions: * The abstract can be improved. There is a lack of information about  "what is replaced by a surrogate model" ( oracles?) or "why a surrogate model is used".

* There is no definitions or intuitive explanations about what is reverse optimization. This term seems  to be important in the main text.

* In algorithm 2, $z_k^{(i)}$ are samples of random variables. How can samples be further optimized by gradient-descent?
soundness: 1
strengths: * A new criterion for candidate molecule generation during active learning is proposed.
* The author implements diverse baseline methods for comparison with the proposed one and considers multiple sources of oracles.
* Implementation code is provided as proof of reproducibility.
summary: This paper proposes a multi-fidelity method that integrates the generative modeling of the molecules and the surrogate modeling of the oracles and aims at a trade-off between the docking oracle that is of low fidelity but of cheap computational cost and binding free energy oracle that is of high fidelity but of expensive computational cost. Besides, the author proposes a new criterion during active learning for generating candidate query molecules to the oracles.
weaknesses: *  The soundness and novelty of the main feature of ML-LAL - " using separate latent spaces and decoders specialized for each fidelity improves surrogate modeling and inter-fidelity information passing"- is questionable. The detailed explanation of my concerns is as follows:
    * The author use a set of random variables $z_{1:K}$, corresponding to $K$ fidelity,  as the latent representation of  molecule $x$. According to the main text, the variational distribution $q(z_{1:K}|x)$ (that approximate true  posterior  $p(z_{1:K}|x)$) is defined by,  $q(z_{k}|z_{k-1})=\mathcal{N}(\mu_k(z_{k-1}),\sigma_k(z_{k-1}))$ where  $z_0=x$ and $k=1,\dots,K$. Enhancing the expressive capbility of  $\mu_k$ and $\sigma_k$  by choosing them the outputs of  neural network $h_k$, the author expects better approximation of $p(z_{1:K}|x)$, thereby so-called "better inter-fidelity information passing". 
        * Defining $F_k(\cdot)=\mu_k(\cdot)+\sqrt{\sigma(\cdot)} \epsilon_k$ with $\epsilon_k\sim \mathcal{N}(0,1)$ and $F'(F(\cdot))=(F'\circ F)(\cdot)$,  then $z_k$ in this paper is represented by $z_k=\mu((F_{k-1}\circ \ldots \circ F_1)(x))+\sqrt{\sigma((F_{k-1}\circ \ldots \circ F_1)(x))} \epsilon_k$.
     * However, we can alternatively represent $z_k$ as $z_k=\mu((h_{k-1}\circ \ldots \circ h_1)(x))+\sqrt{\sigma((h_{k-1}\circ \ldots \circ h_1)(x))} \epsilon_k$ ($h' \circ h$ means stacking the two neural networks). This way, I believe, also enhances the expressive capability of  $\mu_k$ and $\sigma_k$.
        *  Since $\epsilon_k$ is non-informative, the issue here is that the method considered in this paper, and the method I provide above are conceptually similar, and the latter one means nothing but adding more neural network layers to $\mu_k$ and $\sigma_k$  as $k$ increases, rendering the novelty of the proposed method trivial, and the claimed ''better message passing''  questionable.

* Following the setup of $q(z_k|x)$, the author propose algorithm 2 for efficiently generation of molecules candidates, which is expected to     be of high scores for each fidelity.   However, if we follows the conceptually similar setup I described above, algorithm 2 just reduces to getting a sample batch $\\{\hat{\epsilon}^{(i)}\\}$ from $\mathcal{N}(0,1)$, and only keeping  the samples $\hat{\epsilon}$ such that  ${z_k}(\hat{\epsilon})$ is of high scores for each $k$.

* The experiment results are not convincing. The detailed reason is as follows:
    * As acknowledged by the author,  the  goal of multi-fidelity methods is to balance the trade-off between fidelity and computation cost under a fixed computation budget (lines 69-70, algorithm 1).  In this paper, the first oracle, docking activity, is computationally inexpensive but has low fidelity, whereas the second oracle, binding free energy, is computationally expensive but offers higher fidelity. 
    * In this sense, we would like to see a training curve as done in the baseline method[1], where the $y$-axis represents the values of performance metrics, and $x$-axis represents the computation cost until the budget is exceeded. 
   * However, the author only provides Table 1 in this paper, which summarizes the final performance metric values. This alone is insufficient to support the claimed capability for the trade-off.

* The other claimed feature- "MF-LAL combines the generative and multi-fidelity surrogate models into a single framework(lines 20-21)"- is questionable.
   * While the author claims that workflow in [1] is *separate* (lines 56-58), we note that  the *integrated* workflow of  MF-LAL is similar to [1]. The *integrated* part, which I found in MF-LAL is that in equation (1),  the first term (corresponding to generative part) and the second term (corresponding to surrogate part) are optimized jointly. 
  * However,  since the two terms are **sequential**, where the output of generation part is the input of the surrogate part, there is no big difference whether we optimize it sequentially as described in [1] or jointly as in MF-LAL.  
  * By the way,  **GPs are non-parametric models**[3], and $\lambda_k$ in line 205 is the hyper-parameters of the mean or covariance function. Thus, the second term does not exist when there are no hyper-parameters or hyper-parameters are fixed.  The surrogate modeling of the oracle is commonly referred to as doing posterior inference of $P(f|x,\mathcal{D})$, where $\mathcal{D}$ is the multi-fidelity dataset, rather than fitting the hyper-parameters[4]. 


* Equation (1) is incorrect.  ELBO (Evidence Lower Bound Objective) is the lower bound of log-likelihood of observations, this is $\log p(x)$. So, for example, the first term for $k=1$ should be $\log p(x)=\log E_{z_1}[\frac{P(x|z_1)P(z_1)}{q(z_1|x)}]\geq E_{z_1}[\log \frac{P(x|z_1)P(z_1)}{q(z_1|x)}]$ rather than $E_{z_1}[\log \frac{P(x|z_1)}{q(z_1|x)}]$.  Besides, if $f_k$ in the second term, is a GP (Gaussian Process),  what is the definition of $p(y|f_k)$ ?  (Is $y$ equal to the output of a GP plus additional Gaussian noise? )

* According the Algorithm 1,  the hyper-parameter $\gamma_k$ is crucial as it dictates whether queries are made at the next level of fidelity. Therefore, an ablation study or discussion for how $\gamma_k$ are selected should be provided. 


[1] Alex Hernandez-Garcia, Nikita Saxena, Moksh Jain, Cheng-Hao Liu, and Yoshua Bengio. Multifidelity
active learning with gflownets. arXiv preprint arXiv:2306.11715, 2023.


[2] Kirthevasan Kandasamy, Gautam Dasarathy, Barnabas Poczos, and Jeff Schneider. The multifidelity
multi-armed bandit. Advances in neural information processing systems, 29, 2016.

[3] Seeger M. Gaussian processes for machine learning. International journal of neural systems. 2004 Apr;14(02):69-106.

[4] Frazier PI. A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811. 2018 Jul 8.

Meta-review: Adaptively calling oracles of different fidelity, Multi-Fidelity Latent Space Active Learning (MF-LAL) was proposed for efficient *de novo* drug design. The main contribution is to develop a latent active learning strategy to efficiently query oracles considering the model accuracy and computational budget for molecular design. Experimental results comparing MF-LAL with both single and multi-fidelity baselines were reported in producing compounds with better binding free energy scores.

The reviewers have expressed concerns on mathematical rigor and practical significance of MF-LAL. More comprehensive and principled evaluation experiments were also requested by including more baselines to better establish the significance of the proposed framework. 

The authors may want to consider carefully addressing these concerns to improve the quality of their future submission.

Decision: Reject