Improving black-box optimization in VAE latent space using decoder uncertainty
Authors: Pascal Notin
José Miguel Hernández-Lobato
Yarin Gal
Venue: NeurIPS 2021 Poster (2021)
Keywords: Optimization
Uncertainty
High dimensionality
Discrete data
Variational autoencoder
Generative models
Molecule generation

Abstract: Optimization in the latent space of variational autoencoders is a promising approach to generate high-dimensional discrete objects that maximize an expensive black-box property (e.g., drug-likeness in molecular generation, function approximation with arithmetic expressions). However, existing methods lack robustness as they may decide to explore areas of the latent space for which no data was available during training and where the decoder can be unreliable, leading to the generation of unrealistic or invalid objects. We propose to leverage the epistemic uncertainty of the decoder to guide the optimization process. This is not trivial though, as a naive estimation of uncertainty in the high-dimensional and structured settings we consider would result in high estimator variance. To solve this problem, we introduce an importance sampling-based estimator that provides more robust estimates of epistemic uncertainty. Our uncertainty-guided optimization approach does not require modifications of the model architecture nor the training process. It produces samples with a better trade-off between black-box objective and validity of the generated samples, sometimes improving both simultaneously. We illustrate these advantages across several experimental settings in digit generation, arithmetic expression approximation and molecule generation for drug design.

Improving black-box optimization in VAE latent space using decoder uncertainty
Pascal Notin
José Miguel Hernández-Lobato Department of Engineering University of Cambridge
Department of Computer Science University of Oxford Oxford, UK
Cambridge, UK jmh233@ ac uk
pascal 1 now.nnns.nncoooommmm uk
Yarin Gal
Department of Computer Science
University Oxford Oxford, UK yarinecs. ox. ac uk
Abstract
Optimization in the latent space of variational autoencoders is a promising ap- proach to generate high-dimensional discrete objects that maximize an expensive black-box property (e.g., drug-likeness in molecular generation, function approxi- mation with arithmetic expressions). However, existing methods lack robustness they may decide to explore areas of the latent space for which no data was available during training and where the decoder be unreliable, leading to the generation of unrealistic or invalid objects. We propose to leverage the epistemic uncertainty of the decoder to guide optimization process. This is not trivial though, as naive estimation of uncertainty in the high-dimensional and structured settings we consider would result in high estimator variance. To solve this problem, we introduce an importance campling-based estimator that provides more robust estimates of epistemic uncertainty. Our uncertainty-guin does not require modifications of the model architecture nor the training process. It produces samples with a better trade-off between black-box objective and validity of the generated samples, sometimes improving both simultaneously. We illustrate these advantages across several experimental settings in digit generation, arithmetic expression approximation and molecule generation for drug design.
optimization approach
Introduction
We consider the task of optimizing expensive black-box objective function taking inputs in high-dimensional discrete space. This could be for example finding new molecules for drug design, or automatically generating computer program that matches desired output. Solving this task directly in the original space (e.g., with discrete local search methods such as genetic algorithms) may be challenging given the complex structure and high dimensionality of the data. Recently, Variational autoencoders (VAEs) 2 have been successfully leveraged to model wide range of discrete data modalities from natural language (3). to arithmetic expressions [4] computer programs (5) or molecules 66. By learning a lower- -iver-dimeniionall continuous representation of objects in their latent space, VAEs allow transform the original discrete optimization problem into simpler continuous optimization one in latent space. For example, this can be achieved via Bayesian Optimization in the latent space, or via gradient ascent with a jointly-traines trained neural network predicting the black box
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

property from the latent space representation [6 7. Initial methods in this area have suffered from the fact that the search in latent space may explore areas for which no data was available at train time, and therefore where the decoder network of the VAE will be unreliable []] seemingly good candidate points latent space may be decoded into objects that are invalid, unrealistic or low quality.
Leetstornell dccodings
belove decoder uncataary Brazrid
ameran Acquisttio function with decoder
Black box objective
Acquisition function
Figure 1: Uncertainty-guide optimization in VAE latent space The goal of black-box optimiza- tion latent space to attain regions with high values of the back-box objective after decoding, while avoiding the regions that lead to invalid decodings (left). Standard Bayesian Optimization in latent space may query these suboptimal areas (c.g.. regions on left hand side, center). High decoder uncertainty regions overlap with regions leading to invalid decodings (right), that censoring high uncertainty points helps guiding the optimization towards the most promising latent points.
While several methods have been introduced to promote validity of decoded objects ($2. D.. they either focus on modifying the generative model learning procedure or adapting the decoder architecture to satisfy the syntactic requirements of the data modality of interest. We propose instead to quantify and leverage the uncertainty
decoder network to guide the optimization process latent space (Fig. [ This approach does not require any change the model training nor architecture, and can casily be integrated within several optimization frameworks. results in a better trade-off between the values of the black-box objective and the validity of the newly generated objects, sometimes improving both simultancously.
To be effective, this method requires robust estimates of model uncertainty for high dimensional structured data. Existing methods for uncertainty estimation in this domain often rely on heuristics or make independence assumptions to make computations tractable ($2.2). We demonstrate that such assumptions are not appropriate our setting, and propose new methods for uncertainty estimation in high dimensional structured data instead.
Our contributions as follows:
We introduce an algorithm to quantify the uncertainty of high-dimensional discrete data, and use estimate the uncertainty of the decoder ($3):
We show how the uncertainty of the decoder can be incorporated across several optimization frameworks, including gradient ascent and Bayesian optimization ($4):
We illustrate our approach in a digit generation setting - a simple setup to provide intuition for the method - then quantify its benefits in the more complex tasks of arithmetic expres- sions approximation and molecule generation, covering a diverse set of decoder architectures across experiments (Convolutional, Recurrent and Graph Neural Networks) ($5).
2 Background
2.1 Optimization of high-dimensional discrete objects with generative models
Focusing on molecular generation, Gómez-Bombarelli et al. 6 propose to train VAE learn a distribution over the so-called SMILES representation of molecules (ie. linear sequences of characters) [9], and subsequently perform the optimization in the latent space. Since the SMILES

---

representation follows strict syntactic requirements that are not explicitly enforeed by the generative model, promising points in latent space may be decoded into invalid molecules. To improve the validity of decoded sequences, Kusner et al. 4 and Dai et al. [5] develop task-specific grammar rules into the VAE decoder, focusing on use cases in molecular and computer program generation. However, crafting the corresponding rules requires domain-specifer knowledge, needs to be designed from scratch for each new task, and may not be straightforward to elicit in the first place (c.g.. digit generation example in (. Griffiths and Hernández-Lobato [10] and Liu et al. T propose instead to formulate the problem as a constrained Bayesian Optimization and chance-constrainee optimization task respectively to simultaneously optimize the target property as well as the probability generate valid sequences. These two approaches require nonetheless access to a function that quantifies the validity realism of objects the training data, which is not readily available in many practical applications. A different line of research has focused on representing high-dimensional structured The Junction Tree VAR (JT-VAE) [4] generates systematically valid molecular graphs, by first generating a tree-structured scaffold over a finite set of molecular clusters, and then assembling these clusters back into molecules with message passing network. The MoIDQN [5] casts the optimization problem as reinforcement learning task (double Q-learning), which allows in turn to more naturally extend to simultaneous optimization of different objectives. GraphAF [6] combines the strengths of autoregressive and flow-based approaches to efficiently generate realistic and valid molecular graphs. Lastly. Tripp et al. [17] show that the black- -ox optimization performance can be further enhanced by iteratively retraining the generative model on the points selected during optimization, with weights that depend on their objective function value.
objects as graphs instead 12 [3
Our approach deviates from all the above in that it is is pprresentation-agnoiiie (works with se- craft custom rules constraints, does not
quences or graphs), does not require Nomain-knowledge change the model architecture nor the learning procedure and can be combined with several of these approaches to reach even stronger optimization performance (see 5).
2.2 Quantifying model uncertainty
Adopting a Bayesian viewpoint, the overall uncertainty of a model in a given region of the input space can be broken down into two types of uncertainty [8]
Epistemic uncertainty:
Uncertainty due to lack of knowledge about that particular region of the input space - the posterior predictive distribution is broad in that region due to lack of information that can be reduced by collecting more data;
Uncertainty due to inherent stochasticity/noise in the observations in that region- - collecting additional data would not further reduce that uncertainty.
Aleatoric uncertainty:
We denote input points x, outputs y and the training data as D. The total uncertainty U of a model at an input point x is typically measured by the predictive entropy, ie. the entropy of the predictive posterior distribution P(y|ær, D):
u(x) H(P(vl=, D)) Σ P(y)2, D) log P(ylæ, D)dy.
Denoting P()[D) the posterior distribution over model parameters 8, we can further decompose the predictive entropy V as the sum of two terms:
u(x)=
D)) E Erro 0)) + (H 6))
(2)
Mutual Information M
Expected cutropy E
The first term - the Mutual Information M between model parameters and the prediction y - is a measure of epistemic uncertainty, as it quantifies the magnitude of the change in model parameters that would result from observing y. If the model is uncertain about its prediction for y. the change in model coefficients from observing y should be high. Conversely, if the model is confident about its prediction for y, model parameters will not vary from observing y:
M(x) H(P(s)z, x, D)) E Er(op) (H(P(y)z, 0)).
(3)
The second term the Expected Entropy ε is a measure of the residual uncertainty, ie. the aleatoric uncertainty:
E(x) Er Erro 0)))
(H(P(y)z,
(4)
3

---

In high dimensional settings, an exact estimation of these different quantities is not tractable, therefore, several approximations and heuristics have been introduced. The softmax variance, i.c. the variance of predictions across model parameters, has been shown to approximate epistemic uncertainty well in certain settings [1
In the context of sequential data, the inherent structure in the data generating process often introduces strong dependencies between the output dimensions, e.g., the tokens a generated sentence. cases where there exist weak correlations between tokens, quantifying the different types of uncertainties
above can be made tractable by ignoring these dependencies [22], in which case the predictive entropy for a sequence (3/, V2...., .... VL) may be approximated as the sum of token-level predictive entropies over the tokens:
u(x) = Σ Dy P(ylla, yk-l,, D)]
Σ E Bo [log P(yr|z, yk-1,, D)]. (5)
Unlike the standard expectation definition which integrates over all y, here only y1 is integrated over, and we condition West,, which are obtained from a sample from P(y|æ, D). The process is repeated for several of these samples and average finally computed reduce variance.
While the above has been shown to work well in certain experiments [22]. valuable information being discarded when ignore dependencies across tokens. These dependencies likely to be informative in applications such as the ones considered in $5.3.1 Naive Monte Carlo-based approximations
expected perform poorly in high dimensions since the majority of samples will be in regions with negligible contribution to the sum (Appendix [A). In an active learning setting, Kirsch et al. 23 derive importance sampling-han based estimator of total uncertainty mitigate these issues. In Natural Language Processing, alternative approaches using domain-specific metrics such the BLEU score have been suggested [24), but these are difficult extend other domains.
3 Importance sampling estimator
We consider discrete output points y that belong to nigh-dimensiona. structured object space S (e.g., long sequences, large graphs), cardinality (8]. An exact estimation of the Mutual Information between outcomes and model parameters 0 (equation 3 is impractical because the expectation involves sum over exponentially many possible outcomes for y
In lieu of the heuristics previously discussed, we obtain a principled approximation to the Mutual Information via Monte Carlo estimation using importance sampling, with adequately chosen importance distribution.
We denote (f)) > P(O the learnt approximation to the posterior, and assume we can approximate expectations over model parameters by sampling M independent samples from g()). We can then re-write the Mutual Information M in equation as follows:
M
[S
M(x)) ~ Σ
Ds log ps M
Σ
Σ
Ps.m log Ps.m
Σ
Σ
Pa.m logPsm Ps log Ps
M
Mlys)
where Ps and Ps.m are shorthands, respectively, for Ply = Velx, - the posterior predictive distribution and P(y= va|æ, 0m) the probability of given output y, E given and sample 0, from the approximate posterior distribution over model parameters g()).
We can then obtain tractable approximation to equation 6 via importance sampling:
M(z) = h(y)) ply.) ply.) Ep
h(v)
h(T.)
((y)
N
p(ū.)
with U.- ~ (-), where p is the importance distribution.
We choose the importance distribution to be the approximate posterior predictive defined over the outputs. We generate outcome G. by first sampling set of parameters O from the approximate posterior, and then generating y. from model defined by that set of parameters dg. This distribution

---

end for will sample mostly from regions in S with high probability under the true posterior predictive for input æ. This is in contrast to a naive sum over all possible outcomes y, many of which will have negligible contribution to the sum. This gives rise to an estimator of Mutual information (obtained with Algorithm with lower variance than naive Monte Carlo counterpart (see Appendix A
Algorithm 1: Importance sampling estimator of MI
for 1 to N do
Sample 0n ~ (()) Ds ~ P(ylar, 0 0o) for m
M do
Sample 6m ((0) Compute Pa.m P(y Vol2, 0m)
end for
Compute P.
1 M7 Em Psymi h1 = 1. TV
Em= (Pa. m log Po.m) ) Ps log Ps
end for
Return M(x) = E. ha
4 Uncertainty-guided optimization in VAE latent space
Black box optimization in VAF latent space. We want to optimize the black-box objective O over high dimensional discrete object space S. We train VAE, with encoder g and decoder f. to learn a continuous lower-dimensional embedding of objects in S. The optimization of is then performed in latent space and the best candidates are subsequently decoded into the original space. As discussed in $2.1 this may lead to invalid or unrealistic decodings when the decoder f operates in regions different from the ones seen during training. We propose detect this regime by quantifying the epistemic uncertainty of the decoder for latent points 2 (note the notation change compared to 3 where we used to denote inputs in the general setting): avoiding regions with high epistemic uncertainty for the decoder will make the overall optimization process more efficient by avoiding invalid decodings. We next discuss how optimization process for two approaches commonly used in latent optimization settings.
leverage the uncertainty of the decoder guide the
Bayesian Optimization with uncertainty-awarc surrogate model uncertainty censoring. We first train a surrogate model, e.g., a Gaussian Process (25), to predict O(x) based on its latent representation We then perform Bayesian Optimization using an appropriate acquisition function (e.g., Upper Confidence Bound or Expected Improvement heuristic). There are two main ways to incorporate the decoder uncertainty to guide this process. The first approach consists in training the surrogate model on an objective that penalizes points with high uncertainty (e.g., optimizing O(z) M(2)). Another method is to censor proposal points that would have Mututal Information (M2 above a predefined uncertainty threshold T (e.g., highest value observed on the training data) each step of batch Bayesian Optimization process (see Algorithm| 2
Algorithm 2: Bayesian Optimization with uncertainty censoring
1: Uncertainty threshold T. number of new points to generate N.
2: Sample M points uniformly at random from the train set, with latent tensor Z and property vector P.
3: for 1 to N do
4: Train single task on (Z, P) and generate candidate points (2=) with predicted properties
(fx) B) by sequentially maximizing the Expected Improvement.
5: Compute decoder uncertainty _M(z)) for & [1,
6:
=k e [1, s.t. M(zx) & 1' then
7:
new candidate index k arg max, (fc) s.t. M(Zx) & T:
8:
else
9:
b** = arg (Mg
10:
end if
11: Decode
candidate zμ* and obtain true property P&s of decoded candidate.
(Z, P) Concatenate (2+* Pe*) and (Z, P).
13:
Uncertainty-constraal gradient ascent. A common architecture design when performing black- box optimization in latent space is to jointly train the VAE with an auxiliary network h (Fig 9 that
5

---

predicts the value of the black box objective (xx) from the encoding 2 of z in latent space 66, 14] This construct is particularly useful in constrained optimization settings in which we want to perform a local search in latent space to maximize O while remaining close to a known input object. The joint training consists of optimizing the sum of the VAF loss (i.e., the ELBO) and the loss from the black-box objective prediction (e.g., MSE loss for continuous output O) via gradient descent, backpropagating gradients through the entire architecture. To optimize objects under this framework (see Algorithm 3) we start from set of initial points in latent space 2 either random sample of latent points, or a subset of points I that we encode in the latent space (z = (()). We then compute the gradient V 2 of the auxiliary network with respect to and perform gradient ascent 2 - + a vgh. We repeat this process a few times until satisfying a stopping criteria (e.g., threshold on predicted values h(2) after fixed number of gradient updates). Finally, we decode the latest latent positions to obtain the set of candidates I = f(z) and measure their actual properties (()). We can further improve the quality of the candidate by censoring the moves in latent during gradient ascent that would result in a value of uncertainty above a predefined threshold T.
Algorithm 3:
gradient ascent
1: Uncertainty threshold T, number of gradient updates N. gradient scale a.
2: Sample M points from the train set, with latent tensor Z = (#x ]e[[,, M1 and property vector P.
3: Compute v.h(2) where h is the auxiliary network predicting P from Z.
4: for é 1 to N do
5: for k- to M do
6: if M[= + aV, (zze)) & T then
7:
26 - 26 + ((x)())).
8: end if
9:
end for end for
10:
11:
Decode final positions 7.
12: Ohtain true properties p* of decoded points.
5
Experimental results
After describing the common experimental setting across applications, we demonstrate the effective- ness of using the uncertainty of the decoder to guide the optimization for constrained digit generation. Our objective is to illustrate the concepts introduced above in a simple and intuitive example. We then move on to quantify the benefits of our approach in the more complex cases of arithmetic expression approximation and molecular generation for drug design.
Uncertainty estimators and baselines Across all experiments, we quantify the Mutual Information between outputs and decoder parameters with both the Importance sampling estimator (IS-MI) described in $3 and based on the token independence approximation (TI-MI) described in $2.2 Sampling from model parameters is achieved via Monte Carlo dropout
(26). We compare optimization results with two baselines: the standard approach that fully ignores decoder uncertainty, and an approach in which we censor proposal points with low probability under the prior distribution (standard normal) of the VAES in latent space (referred to as 'NLLP").
Optimization We perform uncertainty-guided optimization in latent space as per the two approaches described in $ For Bayesian Optimization, we train a single task Gaussian Process as our surrogate model based on random subset of training points embedded in latent space and their corresponding black-box objective values. We then perform several iterations of batch Bayesian Optimization using the Expected Improvement heuristic as our acquisition function. At each iteration we select a batch of 20 latent vectors by sequentially maximizing the acquisition function. We select the point with the highest predicted target value for which the decoder uncertainty is below a predefined threshold (c.g.. 99th percentile of decoder uncertainty values observed on the training set) or the one with lowest uncertainty if no point in the batch is below the threshold. We re-train the surrogate model with the newly generated point at each step. For gradient ascent, we randomly sample points from the training set, embed them in latent space, and use these as our starting positions. We then compute the gradient of the auxiliary property network with respect to latent positions and accept proposal moves along these directions if the decoder uncertainty at the corresponding position in latent is below a predefined threshold (c.g., 99th percentile of decoder uncertainty on the training set). For practical

---

considerations, we suggest to always start with high value for the threshold to not unnecessarily constrain the optimization, and move stricter uncertainty constraints if the validity or quality is not high enough for the particular use case considered (see Table [0] for an analysis on the impact of that hyperparameter choice). All optimization experiments reported below are carried out 10 times independently with different random seeds.
5.1 Illustrative example in digit generation
Setup In this first setting, our objective is to generate valid images of the digit 3 that are as thick as possible. We train VAE model generating images of the digit 3 jointly with an auxiliary network predicting their thickness. We use a 'Conv-Decony' [27] architecture for the VAE and a 3-layer feedforward network for property prediction. The underlying data consist of grayscale images of the digit 3 extracted from the MNIST dataset
(28) that we discretize to form tensors of binary values. We the sum of pixel intensities across given image a proxy for the thickness of the corresponding digit. An unconstrained optimization in the latent space would ultimately lead to the generation of invalid white 'blobs'. To avoid this failure mode and promote validity of the resulting candidate set, we leverage the uncertainty of the decoder network. In order to assess the validity of objects, we independently train deep Convolutional Neural Network classify images of the digit (binary classification).
Results Latent points with high decoder uncertainty lead to higher rate of invalid decoded digits (Appendix B.3) which help avoid the aforementioned failure mode during optimization. For both Bayesian Optimization and gradient ascent, observe that the decoder uncertainty constraints help ensuring the generated digits remain valid, while preserving the ability of the optimization algorithm to increase the thickness (Fig 2
Starting points
3
3
3
3
3
Method
Top 10 avg.
Validity (%) 1
Grad ent ascont with no uncortainty
No uncertainty
-0.96 + 0.04
77% + 0.6%
3
NLLP
-0.97 1 0.05
76% 1 0.8%
TI-MI
-0.72 1 0.06
96% 1 0.5%
Grad ent ascent with decoder uncertainty
IS-MI
-0.70 0.04
98% + 0.5%
3
Figure 3: Arithmetic expressions approxi- mation results Mutual Information-based con- straints during Bayesian optimization in latent space help promoting higher validity % of de- codings while increasing black-box objective values. Uncertainty threshold values used for censoring candidate points are based on decoder uncertainty values observed on the training data (95th percentile).
Bayesian
Optimizatian in with no uncertainty
3
Bayesian Optimization with decoder uncertainty
Figure 2: Top 5 decoded digits after optimiza- tion Leveraging decoder uncertainty helps pre- venting the generation of invalid digits.
5.2 Arithmetic expressions approximation
Setup We follow an experimental design similar to Kusner et al. 4 in which we seek to optimize univariate arithmetic expressions generated by formal grammar (rules and examples are provided in Appendix C). The objective is to find expression that minimizes the mean squared error (MSE) with respect to predefined target expression (1/3 * x sin(æ * x) x)). More specifically, since the presence of exponentials in expressions may results in very large MSE values, the black-box box objective defined as -log(1 + MSE). We train a 'Character VAE' (CVAE) 4 on 80, 000 expressions generated by the formal grammar, then perform optimization in the latent space.
Results We observe that methods leveraging the decoder uncertainty result almost always valid decodings, and reach higher average values of the black box objective for valid decoded expressions
compared to baselines (Fig. 3 gradient ascent results Appendix (.4). In particular, censoring
candidate points based on their probability under the standard normal prior does not help promoting
7

---

validity of decodings at all. In this setting with relatively short sequences (arithmetic expressions have most 19 characters), leveraging the TI-MI or IS-MI estimators leads to comparable performance.
5.3 Molecule generation for drug discovery
Molecular generation for drug design seeks to identify new molecules satisfying desired chemical properties. Molecules are typically either represented sequences of characters, using their SMILES representation [9], or as graphs of atoms [12]. We demonstrate the effectiveness of the approach described in for these two different representations: experiments with the 'Character VAE' (CVAE) for molecules [6] leverage the SMILES representation, while experiments with the JT-V [1] are based on a graph representation of molecules. For both architectures, we trained our models on a set of 250k drug-like molecules from the ZINC dataset 29
5.3.1 Character VAE (CVAE)
Setup We jointly train a CVAE model, which learns to encode and decode molecules SMILES strings, along with an auxiliary property network that predicts a target property of these molccules. Following prior work 4 5 [4 we define the black-box objective the octanol-water partition coefficient penalized by the synthetic accessibility score and the number of long cycles, (Appendix [.I] and we refer to it as 'Penalized logP' for brevity. Since the SMILES representation of molecules follows strict syntax that determines whether a given expression is valid or not, we are interested in generating molecules that simultaneously maximize the target property and represent valid SMILES expressions.
Peeallord logs
Decaddrr uncertainty 05-28)
sall decadi
Laberlypee Dimemion
10
Latent Dinerdon
Deertspoce-iierrdoo Dimerdoon
Latentspace Dimenslon
Figure 4: CVAE latent space visualization. We apply Principal Component Analysis the embedding of the full training data and keep the first 2 components. We then create grid on the resulting 2D-space and measure the penalized logP (a), decoder uncertainty (b) and the proportion of valid decodings in that region (c) (a & с averaged over 300 decodings; b measured via IS by sampling 100 times from the importance distribution, and averaging over 100 samples of model parameters.; for a, white squares correspond to regions where none of the 300 decodings are valid). We observe a strong overlap between decoder uncertainty (b) and validity of decodings (c).
Results We first verify that our estimator is able to discriminate points in ni-distribbuion (low uncertainty) vs dist---iiittiittonnn (high uncertainty). We consider 4 distinct sets of points in latent: embeddings into latent space of a random sample from the train and test sets, random samples from the VAE prior (standard normal) and random samples "far from the prior" (we sample from an isotropic gaussian with standard deviation equal to 10). As can be seen on Fig uncertainty estimates for the first sets strongly overlap while being disjoint from the estimates corresponding to points far from the prior. Furthermore, we observe a strong correlation between low decoder uncertainty and regions that lead to valid SMILES decodings (Fig. 4). This is corroborated by the analysis described in Fig, 5e: when considering latent points "far from the prior", points for which the decoder uncertainty is lower than predefined threshold (e.g., maximum value observed on training data) will lead to a significantly higher proportion of valid decoded molecules compared to latent points with uncertainty above the threshold. This is critical it allows to censor points that will likely lead to invalid decodings, even when we move far from the prior in latent space.
For the Bayesian Optimization experiments, we investigate the impact of different bounds on the space we optimize within, as well as different uncertainty thresholds. As we increase the bounds, we typically reach higher optima, at the cost of higher fraction of invalid decodings during search. We
8

---

Micse fresholi
9h of water deccelings
% of vaite decoddings
Denstyy (t))
10
Docodar uncertainty US1
defainetnncd molocules based on uncortaliiy
Stancard deviation used for sampling
Figure 5: Uncertainty estimator. a) Distribution of decoder uncertainty values (IS-MI) for Ik samples for 4 distinct sets (train & test set samples embedded in latent space; samples from the prior; samples far from the prior). b) Valid decodings (%) as a function of the proportion of samples kept based their uncertainty 50% samples from test set 50% of samples far from the prior). The IS-MI estimator has superior ability to identify points leading to invalid decodings. normal distribution with increasing standard deviation. Samples with decoder uncertainty below predefined threshold (maximum IS-MI value observed on training data) have much higher rate of valid decodings. Points above the threshold are very likely to lead to invalid decodings.
eliminating points with high uncertainty first (dataset comprised of
Valid decodings (%) for samples from
Table I: CVAE Bayesian Optimization results. Censoring proposal points with high decoder uncertainty values with the IS-MI estimator helps increase validity across experiments. As increase the bounds on the Bayesian Optimization search space, validity % generally decreases but remains 5- 1-10x higher when leveraging IS-MI compared to baselines. This is critical as is helps uncover molecules with very high penalized logP values.
Search bounds
Decoder
Penalized logP
Validity (%)
uncertainty
Top 1
Avg. top 10
5
None
4.0 0.2
2.5 0.2
22% + 1.4%
NLLP
4.2 + 0.2
2.7 0.1
30% + 1.3%
TI-MI
2.3 + 0.1
21% + 0.8%
IS-MI
4.5 0.2
3.0 0.1
33% + 1.8%
10
None
3.9 1.2
-23
1% + 0.4%
NLLP
2.9
0.5
3% + 0.7%
TI-MI
5.9 3.6
1.1 1.5
2% + 0.4%
IS-MI
6.6 + 0.6
1.6 + 0.8
11 11% + 0.8%
15
None
10.3 + 4.3
5.0 + 2.6
1% + 0.3%
NLLP
3.9 + 2.5
0.8 + 1.2
1% + 0.3%
TI-MI
6.7 3.8
6.4 3.9
1% 0.3%
IS-MI
27.6 +
2.2
9.9 - 1.3
5% 0.7%
obtain higher validity % and penalized logP values when leveraging the decoder uncertainty (Table D In this setting, the token-level independence assumption (TI-MI) leads to poor performance compared to the importance sampling-basee based estimator (IS-MI). Results also robust to the choice of decoder uncertainty thresholds (D.4).
5.3.2 Junction Tree VAE (JT-VAE)
Setup We train Junction Tree VAE model (JT-VAE) 14] using the same dataset of 250k molecules (ZINC) and black-box objective (penalized logP) as for the CVAE experiments. All molecules generated by the JT-VAE are valid by design. However, high quality, which we assess with the quality filters proposed by Brown et al. [0] that aim at ruling out "compounds which are potentially unstable, reactive, laborious to synthesize, or simply unpleasant to the eye of medicinal chemists." We show that is straightforward attain state-of-the- art performance in terms of penalized logP values with the basic optimization approaches described in 4 moving sufficiently 'far away' in latent, but that in doing tend to generate molecules
all generated molecules will be of

---

Table 2: JT-VAE Gradient ascent results. We obtain state-of-the-art performance in terms of penalized logp via gradient ascent. However, most generated molecules are of very low quality (only 1% pass the quality filters from Brown et al. (30). Leveraging the uncertainty of the decoder (IS-MI) during optimization helps generating molecules with high penalized logP and high quality. NLLP constraints help maintain high quality but Iead to suboptimal black-box objective values. Results with different hyperparameters and threshold values for each method are reported in Table 13
Decoder
Penalized logP- - Before filters
Quality top 10 (%)
Penalized logP Passing filters
uncertainty
Top 1
Avg. top 10 1
Top 1
Avg. top
None
23.7 + 1.3
17.0 + 0.6 2.5 0.1
1% + 1%
1.2 + 1.2
0.3 + 0.3
NLLP
3.0 0.1
82% 1 6%
3.0 1 0.1
2.0 0.2
IS-MI
8.4 10.8
6.0 1 0.3
89% 1 3%
7.7 1 0.7
5,3 1 0.3
that never pass quality filters. Factoring in decoder uncertainty during optimization helps generate new molecules with both high penalized logP values and high quality.
Using notations from $ sampling a new object y. is achieved by successively decoding from the junction tree decoder and then the graph decoder. We then decompose log Ps,m the log probability the different
of the sampled graph molecule predictions made by the junction tree decoder and graph decoder, namely the topology and node predictions in the junction tree decoder, and the subgraph prediction in the graph decoder.
the sum of the log probabilities corresponding
We replicate the analysis described in [.3.]]] with the 4 distinct datasets in latent space (i.e., train, test, prior and far from prior) and observe similar results: the histogram of decoder uncertainty values for points "far from the prior" is disjoint from the other three histograms (Appendix E.3). confirming the ability of the estimator to identify
points.
Results Both gradient ascent (Table 2 and Bayesian Optimization (Appendix E.44 allow to generate new molecules with state-of-the-art performance in terms of penalized logP (Table 12 However, the majority of these molecules do not pass quality filters. Leveraging decoder uncerfainty leads to the generation of high logP and high quality molecules. Using likelihood under the prior (NLLP) to achieve the same is detrimental to optimization performance.
6 Discussion and conclusion
Strengths Leveraging the uncertainty of the decoder is a simple yet effective approach to promote the validity or quality of objects generated while optimizing given black-box property in VAE latent space. It is model architecture-agnosiic and does not require model re-training. The only requirement is the ability to sample from decoder parameters which we have achieved in this work via Monte Carlo dropout given its practical simplicity. In several of the experimental settings, using the decoder uncertainty also helped attain substantially higher values of the black-box objective during optimization (S 1.3.1 I in particular), as the optimization procedure was effectively guided to avoid exploring regions systematically leading to invalid decodings and focusing on more promising regions instead. I Lastly, the importance campling-based estimator introduced in this work is general-purpose and may be relevant to other applications that would benefit from reliable epistemic uncertainty estimates for complex high-dimensional data (e.g., active learning or anomaly detection).
Limitations While the algorithm for estimator is easily parallelizable, it nonetheless results in computational overhead at each optimization step (which depends on several factors eg., number of samples, decoder architecture, hardware used). In the settings considered in our work (eg., molecular generation). the costs resulting from this overhead are however negligible in comparison to the costs stemming from evaluating the black-box objective (eg., wet lab experiment, expensive simulator).
Future directions Developing an approach to backpropagate through the decoder uncertainty esti- mator may provide compelling alternatives algorithms presented in $ Leveraging the uncertainty of the property network jointly with constraints on the decoder uncertainty may make the search in latent even more robust to invalid decodings. Lastly, our approach may also be used jointly with approaches performing weighted VAE re-training during optimization [17, potentially leading to stronger optimization performance.
10

---

Acknowledgments
P.N. is supported by GSK and the UK Engineering and Physical Sciences Research Council (EPSRC ICASE award no. 18000077). J.M.H.-II. acknowledges support from Turing Al Fellowship under grant EP/V023756/1. Y.G. holds a Turing Al Fellowship (Phase 1) at the Alan Turing Institute, which is supported by EPSRC grant reference V030302/1.
References
[1] Diederik P. Kingma and Max Welling. Auto-encoding variational hayes. CoRR, ahs/ 14, 2014.
[2] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approxi- mate inference in deep generative models. In /CML, 2014.
[3] Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. Generating sentenees from a continuous space. In CoNIJ, 2016.
[4] Matt J. Kusner, Brooks Paige, and José Miguel Hernández-Lobato. Grammar variational autoencoder. In ICML, 2017.
[5] Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax- variational autoencoder for structured data. In ICLR, 2018.
[6] Rafael Bonnez-lo Jennifer N. Wei, David Duvenaud, José Miguel Hemández-Lohate Benjamín Sáncher-Lengeling, Dennis Sheherla, Jorge Alán Aspuru-Guza Automatic chemical design using data continuous representation of molecules. ACS Central Science, 4(2):268 276. Jan 2018. ISSN 2374-7951. doi: 10. 10.10021/acccentee1700057222 URL
Timothy D. Hirzel, Ryan P. Adams, and
doi org 0221 wttp:/...2222accceeeeeeeee
7bC0572
[7] John Bradshaw, Brooks Paige, Matt J. Kusner, Marwin H. S. Segler, and José Miguel Hernández-Lobato. model to search syuthesizable molecules. In NewrIPS, 2019.
[8] David Janz, Jos der Westhuizen, and José Miguel Heeráádezz Actively learning what makes discrete sequence valid, 2017.
[9] David Weininger. Smiles, chemical language and information system. 1. introduction to methodology and encoding rules. J Chem Inf Compur Sci, 1--36,,,,,, 1988. doi: 10. 10.1021/ci0005570005 URI.
http://dx. doi www.ooo/oo.oooooooooo 10. /2//ci 00057a005)
[10] Ryan- Ryan-Rhys Griffiths and José Miguel Hernández- chemical design using variational autoencoders. Chemical Science, 2020.
Constrained bayesian optimization for automatic
[11] Xianggen Liu, Jian Peng. Qiang Liu, and Sen Song. chance-constrained generative framework for sequence optimization. In ICML, 2020.
[12] David Duvenaud, Dougal Maclaurin, Jorge Spallara-P,rrrauirrre, Rafael Kommez-ombarelli, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P. Adams. Convolutional networks on graphs for learning molecular fingerprints. In Advances in Neural Information Processing Systems, 2015.
[13] Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. Learning deep generative models of graphs. In ICLR, 2018.
[14] Wengong Jin, Regina Barzilay. and T. Jaakkola. Junction tree variational autoencoder for molecular graph generation. In JCML, 2018
[15] Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N. Zare, and Patrick Riley. Optimization molecules via deep reinforcement learning. Scientific Reports, 9(1). Jul 2019. ISSN 2042-2322. doi: 1031/ $41598-019-471 x UtLhttp:// //x doi. erg/ 10.00000 10311108/5500 598 019-011991000001 4/148-x
[16] Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow- based autoregressive model for molecular graph generation. In ICLR, 2020.
[17] Austin Tripp. Erik Daxberger, and José Miguel Lornand Sample-efficient optimization in the latent space of deep generative models via weighted retraining. In Advances in Neural Information Processing Systems, 2020.
[18] Alex Kendall and Yarin Gal, What uncertainties do we need in bayesian deep learning for computer vision? In NIPS, 2017.

---

[19] Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. 2017 IEEE Symposium on Security and Privacy (SP), pages 39-57, 2017.
[20] Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, and Andrew B. Gardner. Detecting adversarial samples from artifacts, 2017.
[21] Lewis Smith and Yarin Gal. Understanding measures uncertainty for adversarial example detection. ArXiv, Abss/183300833, 2018.
[22] Andrey Malinin and Mark John Francis Gales. Uncertainty estimation in autoregressive structured prediction. In ICLR, 2021.
[23] Andreas Kirsch, Joost R. van Amersfoort, and Yarin Gal. Batchhald: Efficient and diverse batch acquisition for deep bayesian active learning. In NeurlPS, 2019.
[24] Tim Xiao, Aidan Gomez, and Yarin Gal. Wat heb je gezegd? detecting out-of-distribution translations with variational transformers. Bayesian Deep Learning Workshop at NIPS 2019, 2019.
[25] Andrew Mchutchon and Carl Rasmussen. Gaussian process training with input noise. J. Shawe- Shawe-Taylor, R. Zemel, P. Bartlett, E. Pereira, and K. Q. Weinherger, editors, Advances in Neural Information Processing Syxtems, volume 24. Curran Associates, Inc., 2011. URL http:: / htt.tppss//ppoceeiiggg.o
www.gss...u.pp.... cc/
[26] Yarin Gal and Zoubin Ghahramani. Dropout a bayesian approximation: Representing model uncertainty in deep learning. ICML,, 2015.
[27] Trina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. Beta-VAE: Learning basic visual concepts with a constrained variational framework. 5th International Conference on Learning Representations, ICLR 2017 Conference Truck Proceedings, 2017. ISSN 1078-0874.
[28] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
[29] J. Irwin, T. Sterling, Michael M. Mysinger, Erin S. Bolstad, and R. Coleman. Zine: A free tool to discover chemistry for biology. Journal of Chemical Information and Modeling, 52:1757 1768, 2012.
[30]
Nathan Brown, Marco Fiscato, Marwin H.S. Segler, and Alain C. Vaucher. Guacamol: Benchmarking models for de novo molecular design. Journal of Chemical Information and Modeling, 5963):006-11108 108. Mar 2019. ISSN 1549-960X doi: 10. jttp://as.c.m.b00000
URL doi
aes. . Jeim. 8b00839
[31] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412,6980, 2015.
[32] Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth. Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, and et al. A review of uncertainty guantification in deep learning: Techniques. applications and challenges. Information Fusion, 76:243 297, doi: int.://..fffus2222.... 05.008. URL http:://xx org./////ooooo./0..//////////
Dec 2021. ISSN 1566-2535
1016/ 5. in fus. 2021 1.....0... .05.1.05.0000 008
[33] Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder- decoder approaches. In SSST@ EMNLP, 2014.
[34] Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barabas Poczos, Jeff Schnei- der, and Fric P. Xing. Chembo: Bayesian optimization of small organic molecules with synthesizable recommendations. In AISTATS, 2019.
[35] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf. Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue, E. Fox, and R. Garnett, editors. Advances in Neural Information Processing Systems 32, pages 8024 8035. Curran Associates, Inc., 2019. URL httt:: neurips wwwp:ppppppppppp...pppepepp
9015-pytoree an- http::/ww...ttttt..e...eeeeeeee. Lew.eep--eaaaannngnieeeg pdE
12

---

[36] Maximilian Balandat, Brian Karrer. Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy. BoTorch: Framework for Efficient Monte-Carlo Bayesian Optimization. In Advances Neural Information Procexsing Systems 33, 2020. URL /tLL:tttpp//azaw..r////ee
1910. 06403
Checklist
1. For all authors...
(a) Do main claims made the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See in particular
S modeling
assumptions and forr discussion of the limitations of the approach.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Appendix F
(d) Have you read ethics review guidelines ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all thcoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions nceded reproduce the main experimental results (either in the supplemental material or as URL)? [Yes] We are open sourcing the code in the repository at the following address: httpp:://ithhn: com/ casaa/ppscanncc. in/ uncertaintyy gwwdguddeee
aetimization
(b) Did you specify all the training details (e.g.. data splits, hyperparameters. how they were chosen)? [Yes] Detailed information for datasets and model hyperparameter values used across experiments are detailed in Appendix (one section dedicated to each experimental setting) (reprodccciiiiiiiiiiiii
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes]
(d) Did you include total amount of compute and the type resources used (e.g., type GPUs, internal cluster, or cloud provider)? [Yes] See Appendix [
4. If you using existing assets (e.g.. code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Appendix G
(b) Did you mention license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as URL? [Yes]
(d) Did you discuss whether and how consent obtained from people whose data you're us-
ing/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiable informa- tion or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "qTE2QB6YMEF", "reviewer": "Reviewer_J3j1", "summary": "The authors use importance sampling to estimate the epistemic uncertainty of the decoder, and use this quantity to avoid latent space regions with high uncertainty when generating high-dimensional discrete data.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I believe that if certain concerns are addressed the authors method can be of practical use to important application domains. The paper is in general well-written and well-motivated.\n\nMy main concern with the paper is a fundamental one. The authors method corresponds to censoring the regions of the latent space into which not many encodings were made. While this may understandably result in more valid generated samples, is this not against the idea of using variational autoencoders in the first place? After all, when taken to extreme, the authors approach would turn the VAE to a classical autoencoder, where the latent space has no regularity or smoothness outside the already existing encodings. One of the main advantages of using a VAE is (hopefully) obtaining a 'regular' latent space, thereby allowing the creation of samples that are both novel _and_ valid. I understand that this is not always the case in terms of validity (hence the authors' paper) but the route the authors have taken runs the risk of defeating the purpose of using a VAE in the first place. After all, they might be sacrificing novelty for validity, since an autoencoder could also create very valid samples, but with no novelty at all.\n\nA different approach that I believe the authors can compare their approach to is work on robust VAEs - I believe such approaches might be competitive with the authors' in terms of novelty / validity trade-off, remaining truer to the idea of using VAEs. Such approaches strive to create more 'smooth' and 'regular' latent spaces, example work include [1] and [2]. How would the authors compare their work to those who attempt to create a more smooth latent space?\n\nI believe an empirical comparison would also be called for, both for the existing comparisons and a potential comparison with the approaches mentioned above. I think the authors should discuss any potentially useful metric that would characterize the novelty of the generated samples, and compare their work with previous ones in this respect as well. This comparison would especially be useful when made against robust VAE approaches.\n\nI am looking forward to authors' response and am open to revising my score accordingly.\n\n[1] https://arxiv.org/abs/2102.07559\n[2] https://arxiv.org/abs/2012.03715", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See my comments above regarding limitations. If published in NeurIPS the authors could include a section about potential ethical/societal implications of their work as a generative modeling method in the camera-ready version of their paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "gOttuNRH-ru", "reviewer": "Reviewer_nbh7", "summary": "This paper propose to improve the latent space optimization (LSO) framework through incorporating uncertainty of decoders. The current  latent sace optimization algorithms are based on (a) training an generative autoencoder of candidate solutions for the optimization problem and (b) searching over latent codes that corresponds to high objective function. Searching over latent codes that are unseen during training may lead to the decoder generating invalid outputs. The authors propose to constrain the search process based on (newly proposed) importance sampling-based estimation of decoder uncertainty. Experiments validate the useful-ness of the approach compared to the two baselines: naive LSO and LSO constrained by prior distribution of latent codes. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I believe this paper propose a sensible approach to tackle an important problem. It seems clear that incorporating decoder uncertainty is beneficial to the latent space optimization framework. However, I believe additional experiments should be conducted to evaluate the useful-ness of the proposed importance sampling scheme to estimate decoder uncertainty.\n\nPros:\n- This paper tackels an important problem of the latent space optimization framework that has gained interest. \n- The proposed approach is simple and sensible. \n- Experiments are conducted over various domains to demonstrate its general usage. \n\nCons:\n- My main concern is on the experiments missing baselines for measuring the decoder uncertainty. There are many works that propose algorithms for evaluating uncertainty of (sequential) generative model and detecting out-of-distribution samples. For example, the authors can compare to the algorithms listed in [Ren et al., 2019]. \n-  The authors should cite & discuss their position of their paper with respect to [Griffith et al., 2020] who performs a constrained Bayesian optimization for LSO. \n- In the supplemenent, it is mentioned that the authors use Monte Carlo dropout to estimate posterior distribution of the neural network parameters. I think this is an imporatant detail that should be shown in the main paper. \n- (minor) The authors could provide results on various score functions for the molecular optimization task. There are many scoring functions that are more relavent to the real applications like drug discovery. For example, the authors can consider the GuacaMol benchmark [Brown et al., 2019] or GSK & JNK [Li et al., 2019]\n\nQuestion:\n- I was able to understand that decoder uncertainty helps to generate latent codes that are decoded in a valid & realistic way (ratio of valid & high-quality molecules). However, I am curious what is the explanation for how improving the decoder uncertainty leads to better performance of LSO. \n\n[Ren et al., 2019] Likelihood Ratios for Out-of-Distribution Detection\n[Griffith et al., 2020] Constrained Bayesian optimization for automatic chemical design using variational autoencoders\n[Brown et al., 2019] GuacaMol: Benchmarking Models for De Novo Molecular Design\n[Li et al., 2019] Multi-objective de novo drug design with conditionalgraph generative model", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"ethical_concerns": "There are no ethical issues with this paper.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "HT94pkrBTyh", "reviewer": "Reviewer_VYPR", "summary": "The author(s) note that modelling discrete objects with VAEs that use a continuous latent representation provides an easy way to optimize an auxiliary \"design\" function, which maps discrete data-space objects to a scalar fitness score, via indirectly optimizing the latent space by back propagating \"design\" function gradients through the VAEs decoder. In other words, this method circumvents discrete optimization methods in favor of simpler continuous optimization strategies. The author(s) correctly observe this can be problematic when such an optimization results in values that have low density under the variational aggregate posterior (i.e. latent codes that produce invalid objects since the decoder has never seen them before). The author(s) propose an importance-sampling-based estimate to measure decoder uncertainty as a way to discourage auxiliary \"design\" function optimization from converging on latent codes that decode into invalid objects. The author(s) integrate this uncertainty into both gradient ascent and Bayesian optimization regimes to experimentally showcase their proposals.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Originality:\n- I am unfamiliar with the latest and greatest mutual information estimation techniques. However, I do know for certain there is a large body of related research on MI estimation. Could the author(s) please discuss which aspects of their approach are novel? From lines 103-106, it sounds the decomposition (equations 2-4) is not novel. The paper reads as if importance-sampling these quantities is novel. Can the author(s) please confirm this is the case? I think the paper would benefit on a relevant works related to MI estimation alone, since this is a major contribution of the paper.\n\n### Quality:\n- The author(s) mention a variety of related works the first paragraph of section 2.1. Yet, many of these methods do not appear as experimental baselines in section 5. Their absence makes it difficult to assess the proposals of this paper. In particular, how is a researcher to select between existing methods or the ones proposed?\n\n### Clarity:\n- While each section reads clearly on its own, I was unable to comprehensively understand the implementation of the manuscript's proposals. This difficulty is the principal reason for my low score. I hope the author(s) can address the following issues. \n- Conventional VAE lexicon uses x for data space z for latent space. Section 2.2 uses x for inputs and y for discrete outputs. Wouldn't the inputs be y as well since a VAE reconstructs the input from itself? Or is y reconstructed from x? How do x and y in section 2.2 relate to conventional VAE lexicon? The introduction encourages the reader think a VAE mindset, but the MI deviates from VAE convention.\n- The author(s) want to improve optimization in the VAE latent space, but spend very little time formulating the optimization procedures in section 4. Lines 159-167 talk about several Bayesian Optimization methods but don't tell the reader which one is used.\n- The paragraph spanning 168-183 seemingly discusses how to use a VAE to auxiliary a surrogate design function, yet never really discusses the specifics of how to incorporate the proposed MI estimation other than the vague, \"we can further improve the\nquality of the candidate set by censoring the moves in latent during gradient ascent that would result\nin a value of uncertainty above a predefined threshold.\"\n- The second and third paragraphs of section 5 address my two points above, but are very short on detail. I recommend combing these sections in an effort to provide more mathematical details of the optimization procedure (equations would be nice).\n- The proposal to \"compute the gradient of the auxiliary property network with respect to latent positions and accept proposal moves along these directions if the decoder uncertainty at the corresponding position in latent is below a predefined threshold (e.g., 99th percentile of decoder uncertainty on the training set\" seems problematic if the latent space utilized by the training set is discontinuous, which can and does occur (see, Adversarial Auto-encoders). Are there other ways to incorporate uncertainty that allow for unconnected regions of latent space (other than doing multiple restarts)?\n\n### Significance:\n- The applications of this paper are numerous in computational biology and drug design. If it is state-of-the-art, which is unclear (see above), then it is of great significance to researchers in these fields.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The author(s) do not really address these issues. Can the author(s) please think of times their uncertainty estimation might produce false estimates or when their optimization proposals might avoid less probably but valid regions of latent space that might be associated with a less-represented group? ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "7iwYmqDdCy9", "reviewer": "Reviewer_4uN8", "summary": "The paper considers the problem of black-box optimization of high-dimensional discrete objects where the inputs are embedded into a continuous latent space (typically using a deep generative model) and the optimization process takes place in this latent space. The main hypothesis in the paper is that existing methods for this problem might not be robust because they explore uncertain parts of the latent space with no training data in that region. The paper proposes to leverage the epistemic uncertainty of the decoder (measured in terms of the mutual information between model parameters and outputs) in order to navigate the latent space for black-box optimization. The mutual information is computed via an importance sampling estimate and employed in Bayesian optimization (BO) and gradient based search on the latent space. Experiments are performed on digit generation, arithmetic expression and molecular generation benchmarks.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- One nice thing about the paper is that the proposed approach is complementary to existing methods and can be employed in a drop-in replacement. \n\n\n- Although the importance sampling estimator gives principled uncertainty estimate, it is an extra computational overhead which can be substantial. The estimator requires sampling from the model posterior (accomplished in the paper via MC-dropout) which is already a very challenging and active research problem. Please discuss the implications of this challenging step in computing the estimator. Please provide a computational complexity analysis of using the proposed approach on top of existing methods. Some empirical analysis in terms of raw wall-clock time would be very useful. \n\n\n- The paper proposes two modifications to BO over latent space: censor points above an uncertainty threshold or training GP surrogate model by penalizing points with high uncertainty. However, in BO, the surrogate model's uncertainty is essential in exploring the function. How does the proposed method interact with the uncertainty of the surrogate model. It seems like it might push BO to over-exploit if the uncertainty threshold is not tuned properly. \n\n\n- Please provide a discussion on the criterion for selecting the right uncertainty threshold and how robust the proposed approach is to this parameter. \n\n\n- The experimental comparison in the paper is slightly weak because the baselines are a bit naive. The right method to compare for this setting is Tripp et al. [1] which also proposes a drop-in replacement approach for latent-space optimization. Please add some discussion discussing the pros and cons of the proposed approach with [1].\n\n\n- A minor point but the references section can be improved significantly. Many references are missing even conference names. Please fix that.\n\nReferences\n\n[1] Tripp, A., Daxberger, E., & Hernández-Lobato, J. M. (2020). Sample-efficient optimization in the latent space of deep generative models via weighted retraining. NeurIPS, 2020", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, Societal impact is discussed in the appendix.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Generation of discrete objects from a conditional density (a decoder) with a continuous latent is of interest in many applications.
This work proposes to use  importance sampling to estimate the uncertainty of the decoder, "and use this quantity to avoid latent space regions with high uncertainty when generating high-dimensional discrete data." 

The key contribution of the work is an importance-sampling-based estimate to measure decoder uncertainty.
This is used to avoid latent codes that correspond to invalid objects. 

The paper is found solid and insightful by the reviewers. During the rebuttal, the authors successfully answered several concerns, and provided a detailed performance comparison with prior methods for molecular generation as well as for measuring performance on additional metric to illustrate the performance of the proposed method.

Overall, there is a consensus for acceptance and I also agree with this decision. 

Decision: Accept (Poster)