Conditional generation of molecules from disentangled representations
Authors: Amina Mollaysa
Brooks Paige
Alexandros  Kalousis
Venue:  (2020)
Keywords: molecule generation
disentangling

Abstract: Though machine learning approaches have shown great success in estimating properties of small molecules, the inverse problem of generating molecules with desired properties remains challenging. This difficulty is in part because the set of molecules which have a given property is structurally very diverse. Treating this inverse problem as a conditional distribution estimation task, we draw upon work in learning disentangled representations to learn a conditional distribution over molecules given a desired property, where the molecular structure is encoded in a continuous latent random variable. By including property information as an input factor independent from the structure representation, one can perform conditional molecule generation via a ``style transfer'' process, in which we explicitly set the property to a desired value at generation time. In contrast to existing approaches, we disentangle the latent factors from the property factors using a regularization term which constrains the generated molecules to have the property provided to the  generation network, no matter how the latent factor changes.

Under review as a conference paper at ICLR 2020
CONDITIONAL GENERATION OF MOLECULES FROM DISENTANGLED REPRESENTATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Though machine learning approaches have shown great success in estimating properties of small molecules, the inverse problem of generating molecules with desired properties remains challenging. This difficulty is in part because the set molecules which have given property is structurally very diverse. Treating this inverse problem as a conditional distribution estimation task, we draw upon work in learning disentangled representations to learn conditional distribution over molecules given desired property, where the molecular structure is encoded in a continuous latent random variable. By including property information an input factor independent from the structure representation, one can perform conditional molecule generation via a "style transfer" process, in which we explicitly set the property to a desired value at generation time. In contrast to existing approaches, we disentangle the latent factors from the property factors using a regularization term which constrains the generated molecules to have the property provided to the generation network, no matter how the latent factor changes.
INTRODUCTION
Conditional molecule generation far from being solved. The main challenge the enormous and discrete nature of the molecules space and the fact that molecule properties are highly sensitive to molecular structure (Kirkpatrick & Ellis 2004). Approaches conditional generation are typically two-step, either using model genetic algorithm to generate candidates which are later filtered, or learning a continuous embedding of the discrete molecules and optimizing a real-valued representation space. The former is computationally expensive, the latter performs conditional generation only very obliquely.
We propose conditional generative model that produces candidate molecules which targeting desired property in a single step. This approach builds on work in structured deep generative models (Kingma al., [014] Siddharth al. 2017). which aim learn disentangled representation that factors into observed properties we want to control for, and latent factors that account for the remaining features which are either hard to annotate irrelevant to the properties we wish to optimize.
We derive regularizer for supervised variational autoencoders which exploits property information that we provide as supervision, ensuring that produced molecules adhere to target properties they are conditioned on. We demonstrate the ability of our model to perform accurate conditional molecule generation and sort of "style transfer" on molecules, where latent representation for single molecule can have its target properties perturbed independently of its learnt structural characteristics, allowing direct and efficient generation of candidates for local optimization of molecules.
2 BACKGROUND
Molecule discovery tasks come in two flavors. Global optimization seeks to find molecules that have particular target property. Local optimization starts from molecules which have a desired property while not straying too far from the prototype. There is some overlap in methods used in the two approaches.
some initial molecule and searches for

---

Under review as a conference paper at ICLR 2020
2.1 DEEP GENERATIVE MODELS FOR MOLECULES
Virtual screening methods start from a large database of possible molecules and retain the promising ones (Eckert & Bajorath 2007). as measured by some quality function s(-). Machine learning approaches expand on this by dynamically generating additional candidate molecules; (Segler et al. (2017) uses a stacked LSTM to produce large numbers of novel molecules which have similar characteristics to an existing database.
For properties which are expensive to evaluate, generating large sets of candidate molecules is not particularly useful. More sample-efficient global search can be achieved using Bayesian optimization methods, which use a generative model with a latent space that functions as a continuous repre- sentation of molecules (Gómez-Bombarelli et al. 2016; Kusner et al., (017). Optimization is then carried out over this continuous representation space to find candidates which are expected to have the desired property. Local gradient- gradient-based search can also be applied continuous latent spaces to optimize the latent representation with respect to a target property (Jin et al. 2018: Liu et al. 2018))
A challenge for these latent variable models is to reliably produce valid molecules. Character variational autoencoders (CVAEs) (Gómez Bombarell et al. 2016) generate molecules one character time, and prone syntactic and semantic errors; the grammar- grammar-based variational autoencoder variational autoencoder (SD-VAE) (Dai et al.) 2018) instead operate in the space of context-free and attribute grammars, respectively, ensure syntactic validity. Other work generative models that operates on graph representations (Simonovsky & Komodakis 2018 De Cao & Kipf [2018] Jiin et al. 2018 You al. 2018
(GVAE) (Kusner et al., 2017) and syntax-directee
Liu al. largely
improving the ability to generate valid molecules.
Suppose we are given a training set of pairs D {xx; y:)], ) I,..,, N, where x corresponds to molecules and y represents a value of some properties of the molecule x. Assume the molecules represent an i.i.d, sample from some unknown distribution p(x). which assigns high probability to molecules believed to be useful for a given task. Aside from SSeglr et al. (2017), which has no latent space and thus directly trains via maximum likelihood, these latent variable models are trained by optimizing standard BLOOO objective for variational autoencoders (?). This entails learning a stochastic encoder go (z(x) which maps molecules into a latent space, and stochastic decoder pe(x(z) for reconstructing molecules, by maximizing
[(B, () = Σ (z xx.) Po po (x, D (10(2, (xx)))))
(1)
Notably, the objective is not function of y: most existing generative models with latent variables do not perform direct conditional generation, and approaches for targeted molecule discovery are bolted on to the learnt model. Some, e.g. Kusner et al. (2017), are trained in an "unsupervised" manner, agnostic to any property which later may need to be optimized. Others, e.g. [Gmmz--ombbreh cet al. (2016); Liu et al. (2018), train the autoencoder jointly alongside a function to predict y from z, hoping guide the latent space to be also good for predicting the desired property. A recent exception is Assouul et al. (2018), which learns a deterministic autoencoder where the decoder takes the latent code and the desired property input, using mutual information term in training steer the model towards generating molecules whose target properties match the input. Guimaraes et al. (2017); De Cao & Kipf (2018): You et al. (2018) instead learn generation models optimized towards specific metrics, such as drug- likelieissses and solubility; the major downside is that these models must be retrained each time for new property. In contrast, the autoencoder-bases methods can be re-used to optimize towards any particular value of the property.
2.2 STYLE TRANSFER WITH SUPERVISED VAES
While the latent representations learned through standard VAE models perform well on the task of molecule reconstruction they do not necessarily provide interpretable factorised representations. A disentangled representation gives us additional control on the molecule generation process, allowing us to modify a single property leaving the remaining unaffected (Bengio al. 2013a). In many cases important variation in the data is easy to annotate. For example in the case of molecule datasets we have access to different functional descriptors of the molecules obtained by chemoinformatics software such as RDKit (Landrum). Particularly useful to us here are supervised methods for learning disentangled representations (Kingma et al. 2014 Siddharth et al. 2017). These are distinct from
2

---

Under review as a conference paper at ICLR 2020
unsupervised disentangling approaches such InfoGAN (Chen al. 2016) 3-VAE (Higgins) et al. 2017). which encourages the latent factor to learn disentangled representation by modifying the objective to promote component independence.
We will learn representations that specifically disentangle molecular properties of interest which we may later want modify. Kingma et al. (2014) demonstrates how disentangling can be used to take two MNIST images of different digits, written in different styles, and independently change the digit while holding the style constant. An analogous operation on molecules would involve holding the physical structure of a molecule (its "style") relatively fixed while modifying a salient property. Unlike (say) the style transfer example for the MNIST digits, the conditional distribution of molecules with a particular value of properties might be very diverse; for example, the QED score attempts to measure the drug-likeness of a molecule, and the set of molecules generated high values of this score would hopefully have high probability on a large, varied set of molecules. An essential challenge here is that the property only provides a very weak signal as to the overall structure of the molecule. To account for this diversity, we model the conditional distribution with a latent variable such that po (xly)
(x p(
Disentangling the latent code z from the property y enables style transfer. This is done by taking an initial x, computing the posterior over the latent variable z, and then generating new x' with the property modified to have a target value y', with (x'ly'' x) J' (x y', 2)po 2))o(z(x)dd.
wilbs)
Z
Concretely, this involves fitting a joint generative model of the form pe(x,y: y. 2) pa (xly, z)p(y) Jz()), in which y and z are indepen- dent under the prior, and we assume a unit multivariate normal prior p(z). To infer the latent variable we will use variational dis- tribution ( (x),, which takes the form of a multivariate normal distribution with parameters a nonlinear function of to approxi- mate the true posterior po (z( This objective function
y
Figure I: A demonstration of style transfer
LELBO Φ)
Σ Bao(x,)x,) [log pe (x, y,, v.) - DEL (go (9x(1)))) pP(z)))
(2)
corresponds to learning a supervised VAE (Kingma et al. 2014), and represents a fairly naïve approach to modeling a conditional distribution.
3 CONDITIONAL GENERATION BY DISENTANGLING
Maximizing this conditional ELBO Eq (2) will likely yield good reconstructions of molecules from an embedding (alongside the true property y). but for properties which only weakly inform the generative model there is nothing to enforce that the variable y actually directly has an effect on the generative process. Since the value y is something x, it is completely possible for all information about y to also be encoded in the representation z, in which case there is no guarantee that the learnt likelihood po px(x z) actually takes into account the value of y - in fact, we know it is possible to fit variational autoencoders where the decoder simply has the form po (x|z) and we are relying on the utility of y in reconstructions see any sort of disentangling effect.
know is derived property of the molecule
3.1 CONSTRAINED ELBO
In the case of conditional generation of molecules, we often have access some oracle function (possibly non-diffcrenniaale)) which for any given x outputs a property estimate y. for instance, the chemoinformatics software RDKit (Landrum). Since for conditional generation our ultimate goal is to generate molecule x for any given target property Yo. which then actually has f(x) Yo. we can reframe the problem by introducing hard constraints on the generated values, ie.. restricting to values of in the training set, max LELBO(O, 9.6 subject to Exx [L[((x) y:Jl =1
3

---

Under review as a conference paper at ICLR 2020
dooodor network Pol x/
discrete sampling procedure
Lo (z/x)
2 latent space
go(2,
generated molscule
(h) womniiiiiinnnn
encoder network
sargel property
Appreximats target property: fu(b) BIJ())] y*
RDKit
Non- Mon-difforrantiable property estimation
Figure 2: Setting and modeling pipeline for conditional generation of molecules, with supervision provided via an external property prediction oracle. Red lines correspond to non- con-differentiable components, including both a potentially complex sampling process and the property prediction itself. The blue dashed line corresponds to the approximate property predictor, which aims to predict the expected value of the property from a continuous relaxation, marginalized over the sampling process.
for all I....,N. N. distribution other than one which simply places point mass on the single training x; associated with yı, but we can relax it by considering that (unlike the molecular space x) the property space y is typically smooth, many properties are continuous-vallee
This is an unreasonably hard constraint, unlikely to be satisfied by any
and correspond to coman-interpretaan scale. Following Ma et al. (2018) and Hu et al. (2017), we reframe the constraint as a soft penalty on the ELBO,
A1
C(0, CELBO - o)
Ex (I) y
so that they are consistent with the property prediction, i.c., as we have an oracle function f which enable us to access the property of any generated data, we can explicitly add soft constraint to our loss function to provide explicit guidance for the generative model such that f(x) = y. This constraint is expected to hold for any pair yx we may happen come across, not just those in the training data. We also show optimizing the relaxed constraint is equivalent to maximizing mutual information with the target y; and generated molecule x; for details appendix Section 6.1
3.2 APPROXIMATING THE PROPERTY PREDICTOR
Introducing the regularizer as in Eq. implicitly guides the reconstruction to take into account the property information, such that the reconstructed data should exhibit properties which match the input properties it is conditioned on. However, existing implementations of are often non-differentiable or CPU-bound, and & are discrete samples from a categorical distribution, all of which means the gradient of the regularizer can't ann' flow back the generator. This is outlined in Figure 2 To enable the gradient based methods on GPUs during training and avoid discrete sampling, one approach would be first fit a differentiable approximation to and then use either a Gumbel-softmax relaxation
(Jang et al.) 2016) or tricks like a sttraight-throughe estimator (Bengio et al., 2013b) as a continuous approximation for the discrete samples. Instead, we propose bypassing the discrete sampling step entirely and learning function fa that can map from learned representation of the molecules directly to molecules property (Hu et al. 2017).
To do this, we take as input the last hidden layer of the decoder network which parameterizes pe(x|z, y). denoting this deterministic transformation as go y). For the grammar VAE and the syntax-directed VAE, this last layer h ge (z. y) is the output of recurrent layer that generates logits corresponding to unmasked and unnormalized log probabilities for each character at each position in the string: see Kusner et al. (2017) and Dai et al. (2018) for details the implementation of the somewhat complex sampling process in the decoder. Ideally, fa would estimate the property distribution obtained by marginalizing out the discrete sampling step, with
fw (h- go [z, Yo)) Epp((la.xa) [f
(4)
where we condition on z, and yo refers to arbitrary input target property.
4

---

Under review as a conference paper at ICLR 2020
Assuming the approximation in Eq. 4 we have
(po( fl y:ll² = ppxx) [ () - y.11
Ep(x) (go - y.ll?,
expectation over real-valued variable which does not depend any of the parameters are estimating, meaning we can use a simple path estimate of the gradient with respect to 0, w by exchanging the gradient with the expectation. We thus define regularization term
A1 Σ
Edisent (0,w)
y=ll²
(5)
which can be used as a drop-in replacement for the non-differentiable penalty term in Eq. candidate objective function
yielding
C(0, 6) s (0, = LELBO(O, ()) - Laiisnt (0,w)
(6)
3.3 LEARNING THE PROPERTY ESTIMATOR JOINTLY WITH GENERATIVE MODEL
While one could imagine attempting learn fw jointly with o, 0 by direct optimization of Eq. C in practice this is very unstable, as values of (e y:) early in training may correspond to very poor generated molecules & which may not have properties all similar yı. This can be sidestepped by training the property estimator jointly as part of an extended generative model on [x
We note that the property estimator far parameterizes probability distribution Pa (x(x)) yo), where x rv pe [x] zz Vo) and f is the oracle function that fxx = y. With Gaussian distribution over the error, we can consider
= (N( (z)))
for small, fixed X2- Therefore, we propose defining a new ELBO based on joint autoencoder for {f(x). y:)). albeit with a factorization such that the input y; bypasses the encoder and is passed directly into the decoder, with joint likelihood
P[0,w) (xi, [xx:) (xx))2z, y+)
pw (J (x)
(8)
This yields a joint ELBO for the training set of
N
[og (z) v:) vz)p(z)
0))
Σ
Bg (x)xx))))
(9)
gol (z(2 xx)
Note that we rewrite this ELBO function of the previous one, with
A2 Σ
CELBO(W,0,O) (ELBO
1ll -vele.
(10)
==1
where we also see that LELBO(W,0,g < LELBO (0, Φ). allowing to define an objective
İ(w. 0.00 x) LELBO 0, x) Laisens (0,w),
(11)
which is a lower bound on Eq. 66) Notice the two terms we have added to the original ELBO are quite similar, differing only in choice of distribution: for learning fa, we wish values of simulated form the approximate posterior 4s 4z(z)) (z|x), x), whereas for enforcing a constraint across all possible generations simulate from the prior p(z).
3.4 GRADIENT ESTIMATION
As the regularizer Ldisene (6, w) encourages disentangling by constraining the molecules generated from y; to have property y; no matter what value z takes, we found that it does not necessarily evaluate at meaningful values of when sampled randomly from p(z). This roughly corresponds to the notion that not all combinations of "style" and property are physically attainable; ideally for style transfer would like the generated molecule to stay "close" in structure the original molecule that we intended to modify. When estimating (gradients of) the soft constraint term Caissen w). found it advantageous use samples of z which correspond to encodings of actual data points, as
5

---

Under review as a conference paper at ICLR 2020
QM9
ZINC
Model
Reconstruciion %
Valid'
Unique &
Novel 16
Recersirucion f
Valid'
Unique %
Novel %
CVAE Gómez. Bombarelli
2016
3,61
10.30
90.0
44.6
0.70
100
Kusner
96.00
601.20
53.70
SD-VAF or 401h
97.84
98,44
99.28
91.97
76.20
43.50
Sup VAE-I-GUU
97.53
93.66
91.30
92.05
74.12
32.81
91.61
100
CGD.
$7.87
Sup VAE:3-GUU
97.81
97.90
95.00 95.09
89.47
$2.40
36.
86.26
CGD-VAE-3-GRU
99.31
97,90
98.77
96.21
$1.80
37.78
98.75
100
Table Reconstruction performance and generation quality (Valid, Unique, Novel).
opposed to random samples from the prior. We approximate expectations with respect to p(x) by looking at the so-called marginal posterior; note that
p(z) (z)
pe (2o
Exo(ziz)) Σ 4s (z|xx),
where the first approximation uses the empirical data distribution as an approximation to the model marginal (x), and the second uses variational posterior approximation qo (z(x)) We define this quantity as q()) N Ej 90 (z(x,), a mixture of Gaussians, which we can sample from by drawing random values from our dataset and then drawing from their encoding distributions.
When we use this estimating gradients of the soft constraint, we can use samples from the same minibatch, exactly corresponding to property transfer task. That is, for any particular y: in the dataset, we can estimate
Ep(x) V fa (go [z, y:) - yvll Ix)),) (owllluu(90zz. - y:ll?.
for any uniformly randomly sampled / i. By sampling zj from g(zz|x,) where j + i, we make sure that all the label information decoder is receiving comes from the actual y, that is feed to the decoder and z, does not include any information about label. This can be evaluated easily by simply evaluating the penalty term of Eq. (10) twice per minibatch; once as in Eq. (TO), and once to approximate Lutisent (0, w) by permuting the properties in the minibatch be assigned to incorrect molecules. We detail the training algorithm Section 6.2 of the appendix.
4 EXPERIMENTS
We experiment with the QM9 dataset (Ramakrishnan et al., (014). that contains 134k molecules with up to 9 heavy atoms, and the ZINC dataset (Sterling & Irwin 2015) containing 250k drug- like molecules. Our goal here is two-fold: we would like to understand (1) whether a supervised variational autoencoder is capable of learning suitable conditional distributions over molecules, and (2) to what extent this task is assisted by the additional regularization term corresponding to the soft constraint.
We represent molecules using the one-hot encoding of their SMILES production rules (Kusner et al. 2017) and add semantic constraint (Dai al. 2018) on the decoder network avoid generating syntactically correct but semantically invalid molecules. We use 80 production rules to describe molecules and set the maximum SMILES sequence length 100 for the QM9 dataset and 278 for the Zinc dataset. We experiment with the logP property of the molecules (Wildman & Crippen] 1999). We use the same encoder and decoder network structure as Dai al. (2018) with the only difference that our decoder takes as input the concatenation of y, z. We give the details of the architecture in the appendix section 6.2
We evaluate the reconstruction accuracy and the quality of the molecules generated by our method, which we denote by CGD- VAE (conditional generation with disentangling) and compare against CVAE (Gómez-Bombarde et al. 2016), GVAE (Kusner et al. 2017), and SDDVAE (Dai et al. (018).. We explore its conditional generation performance in two settings: controlling only the property value and controlling both the property value and the molecule structure to what can be seen property transfer. We took the results of CVAE, GVAE from the literature. For SD- VAE we used the authors code with the default values to generate results for QM9 since these were not available for QM9. We also implemented supervised VAE versions of SD- VAE which we denote Sup- VAE-X-AE--RR (X (1, 3), denotes GRU layers) and which can do conditional generation.

---

Under review as a conference paper at ICLR 2020
Wilenute A. Light 5. 1.68338
PACINAAE LIPP (8:3299
+0.779
ocreratee movetute Lish: 6.9112
0.994
0530
0.680
3
0.3076
0.8900
6638
HN
Figure 3: Conditional generation given the de- sired 0055=- row molecules have logP within a 15% range of the desired one.
Figure 4: Property transfer
Before proceeding with the experiments we will give some additional details on how we do conditional generation from (x(()))) given the target property Yo. Instead of marginalizing over the prior p(z), we mirror the approach taken during training and integrate over an approximation to the marginal inference distribution (z(z)
N q (z(2)x)) which better characterizes where the mass of the dataset is in the latent space. However, as N is large and we do not wish to keep the entire dataset available at test time, approximate 9o (z(z)) with an isotropic Gaussian distribution do (z) (z 0, o')). We estimate for each model by Monte Carlo samples from qo (z). For the supervised VAE without the soft constraint regularizer this yields 0.053 for QM9 and O. 18 for ZINC. For our model with the soft constraint we get 0.0354 for QM9 and 0.096 for ZINC. We do conditional generation of x given Yo by sampling from po (x( = 9- (z)po (x Z yo)dz.
We evaluate reconstruction performance in terms of the correctly reconstructed molecules test sets of size 10k for QM9 and 5k for ZINC, for the latter we used the default test set. We evaluate the generated molecules' quality by the percentage of valid, unique (i.e. percentage of unique molecules among the generated valid molecules) and novel (i.e. percentage of molecules never seen in the training set among the generated molecules) molecules. We estimate these quantities by sampling 10k (5K for ZINC) from the qo (z) and coupling each one of them with a logP value, y. randomly selected from the test set, and we subsequently decode the z, y concatenation. We can see that our model has better reconstruction performance compared to the baselines while in some cases generating slightly less valid molecules table terms of the three quality measures achieves an excellent performance across all three metrics being always one of the two best performing methods for any metric.
To visualise how the conditional generation operates we randomly sample from the test set some molecule and obtain its property value yo- We then draw 50 random samples z; from go (z) and decode the [z,,0 vectors. Among the generated valid molecules we compute the percentage of those that have a property value y; that within a 15% range from the Yo property value. In Figure present the molecules obtained for test molecule that had logP of -0.5759. Out of the 50 generated molecules 46 were valid of which we give the five that were within a 15% range from the get molecules that are structurally very different from the original one yet they have similar logP value.
Yo value in Figure 3 As can
To quantify the quality of the conditional generations we measure the correlation between the property value we obtain by the conditional generation and the property value on which we conditioned the generation. We randomly sample 1000 y values from the test set and 1000 values from the approximate learned prior q- (z). We decode each pair, obtain & ~ Pe (x y, z), and then measure the correlation of the original y with the ý of generated x. In Table 2 we give the correlation estimates for our method and the Sup-VAB baselines. As we can see our method has a considerably higher correlation score between the input and the obtained property than Sup-VAF Conditional generation seems considerably harder for the ZINC dataset for all methods.
To visualise the style transfer behavior of our model we randomly sample two molecules A,A X from the test set. We then sample A from the learned posterior 4s (z)xx, x)). We subsequently decode [Z YB), Ys is the property of xB. and get a new molecule & Ideally, the obtained molecule AAB should have a property value (logP) close to the target yæ and be similar to XA. In Figure 4 we give one such example. To put the results into context in Figure in appendix, we give the results of a virtual screening method, where select from the full dataset five molecules which are structurally
7

---

Under review as a conference paper at ICLR 2020
y
3307
223
1366
400
EUC
7.40
2,00 2.800
7.30 2000
7,40 7.00-1160 (.800
7 1.00g
7,4g
7,25 3200)
Z
Figure 5: Style transfer. The of the nine real molecules placed in the x-axis is combined with 11 property values, sampled [-4.9, 4.9], the resulting pair is decoded to molecule.
similar to x,1 and have logP values close to yB- As we can see the molecule that our model generates a new one.
To quantify the style transfer performance we proceed in exactly the same manner as we did to quantify the conditional generation performance. However, now instead of sampling z from the approximate learned prior, 9- (z). we first sample some from the test set and then we sample from the learned posterior q x x). The results are in the second column of Table 2 As we can see the correlation values are now lower than the ones we obtained in the simple conditional generation case. This can be explained by the fact that now we are forcing a specific combination structure, z comes from a real molecule, and property. which might simply be physically infeasible since the molecule space discrete and not all combinations are possible, In addition, as it was the case for the conditional generation, style transfer is considerably more difficult for the ZINC dataset.
further explore the style transfer and visualize how our model covers the combined space of molecule structure and properties. We sample nine molecules from the QM9 test set, and get their encodings. For each such encoding we decode the vectors [z. 3). y E [-4.9, 4.9], with the y (logP) interval sampled at 11 points. We give in Figure 5 resulting valid molecules, each column there corresponds to one of the nine original molecules, the ones surrounded by dotted rectangle, and their decodings with different logP values.For each original molecule ordered along the y axis according to the y property that they actually exhibit. The x-axis does not provide an ordering of the original molecules according to z, in fact we have ordered the original molecules by their property. As we can see not all (z, v) combinations produce result. These holes can be explained either by the physical infeasibility of the combination and/or limitation of the learned model.
give the generated molecules
We can use conditional generation control in a fine manner the value of the desired property, to what can be seen as direct property optimization. We visualise the level of control we have on an experiment with a single molecule (with logP is 1171,,, which we randomly sample from the test set. We obtain its encoding and perform generations with increased logP taking values in 1000 point grid in [-1.137, 4.9]. We then decode [z. y:l and compute the logP value of the generated molccules. Among the 1000 generated molecules only 19 are unique. We get increase of logP of very discrete nature, Figure 6 As already discussed not all combinations of structure and properties possible. The generated molecules themselves shown in the supplemental material.
8

---

Under review as a conference paper at ICLR 2020
Increasing the Loge of given molecule
Model
x ~ (=(2)
x ((4/x)
Sup-VAE--GGRE
0,5420
0,2526
Logp genensted metersies
QM9
CGD-VAE-I-GRU
0.7185
0,5005
Jup-VAF---GRE
0.6958
0,4204
BED-VVE-3-GGE
0.7414
0,4715
Sup-VAE-1-GRE
0.2301
0.0481
CGD-VAE-I-GHE
0.3877
0,0880
ZINC
Sup-VAE-3-GRU
0.3514
0,1808
CGD-VAE-3-GRU 0,3966
0,1559
Table 2: Correlation between the desired input property and the obtained property z ~ Go (z) corresponding to conditional gen- eration), and x, z ((z)x) property trans- fer case.
LogP value used for decoding
Figure 6: Property optimization. Given a start molecule (red), we combine its z with 1000 log? values and decode (blue)
2.00
CSD-VAL-3--0 U-UUUUUUUUGUUU
COD VAE
100
075
050
029
000
-2
0. 02 06 1.0
Figure 7: A comparison of simulated logP values and Tanimoto similarity to a target on the ZINC dataset. While the stacked LSTM model has high accuracy in terms of matching the desired property. would require drawing many samples before finding any close matches to any particular desired prototype. The (GD-VAE-3-GRL model represents middle ground between standard VAE model which does not condition on the property, and the stacked LSTM model which does not learn a reusable representation.
4.1 CONDITIONAL LSTM BASELINE
Finally, we consider variant of the stacked LSTM model Sfgler al. (2017). with no latent space, where the model is modified to take target logP value as an additional input at each generation step. This model forms a very strong baseline for many distribution matching tasks (Liu et al., 2018 ?), though as best we are aware this has never been used directly for conditional generation given target property. We use a modification the implementation provided by settings, and fit the model by maximum likelihood training on pe (x(y) = II" po
(?) with three layers and default
Training on the ZINC dataset, we find the generated molecules from this model have very high correlation 0.975 with the target logP value, greatly outperforming any of the latent variable models we consider. This suggests that such a model would be very useful for generating candidates globally. but as the model has no latent variable it is not amenable to style transfer. We observe this in Figure 7 which samples 100 candidate molecules from both the stacked LSTM model and for GRU, conditioning on the property of one randomly-chosen test set example, while computing the Tanimoto similarity (computed using Morgan fingerprints of radius 2) to a second randomly-chose test set example, across 200 pairs. The VAE has higher Tanimoto similarities as it can condition on the latent variable the target molecule, representing a trade-off against the better adherence to the target property value of the unconditioned LSTM.
5 CONCLUSION
We presented single step approach for the conditional generation of molecules with desired properties. Our model allows also to condition generation on a prototype molecule with a desired high-level structure. This work thus directly inverts the traditional relationship between molecules and their properties. We found that training the deep generative models conditional on target properties, following supervised VAE approach, does not appreciably harm the quality of the unconditional

---

Under review as a conference paper at ICLR 2020
generative model measured by validity, novelty, and uniqueness of samples. Furthermore, see that the additional act of regularizing the output using an approximate property predictor helps improve both reconstruction accuracy and property correlations in most combinations of tasks and datasets, particularly for the smaller QM9 dataset and for smaller models with fewer RNN layers. We also note that although none of the deep latent variable models competitive with LSTTM baseline when purely considering generation conditioned on target property value, the low Tanimoto similarity between randomly sampled candidates and an arbitrary style transfer target makes clear that such model is not suitable for targeted generation of candidates which are close in structure to a particular prototype.
In future work, we want to explore how to further improve the correlation between the desired input properties to the decoder, and the properties of the generated molecules. Moreover, we want also to condition multiple properties; while this is in principle possible in our framework, we do not explore empirically here. Modifying a single property while constraining the remaining to close to the original can further aggravate the infeasibility problem, not all combinations of molecular properties may even be feasible, perhaps requiring learning a dependency structure between multiple properties.
REFERENCES
Rim Assouel, Mohamed Ahmed, Marwin H. Segler. Amir Saffari, and Yoshua Bengio. Defactor: Differentiable edge factorization-basee probabilistic graph generation. CoRR, abs/1811.09766. 2018.
wwwpa/xbi.rrr.oooaaoaaaaaaaaaaaa /1111, 69766
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions pattern analysis and machine intelligence, 35(8):1798-1) 1828, 2013a.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arxiv:1308.343:: 2013b.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in neural information processing systems, pp. 2172-2180, 2016.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational autoencoder for structured data. arXiv preprint arXiv:1802.08738 2018.
Nicola De Cao and Thomas Kipf. MoIGAN: An implicit generative model for small molecular graphs. May 2018.
Hanna Eckert and Jürgen Bajorath. Molecular similarity analysis in virtual screening: foundations, limitations and novel approaches. Drug discovery today, 12(5-6):225-233, 2007.
Rafael #6öme--Bombarele David K. Duvenaud, José Miguel Hernández-Lobato, Jorge Aguilera- Iparraguirre, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik. Automatic chemical design using data-driven continuous representation of molecules. CoRR, abs/1610.02415, 2016.
URL http:httpp//ii.....// http://ooggaabaa.a.eess 2001///00//111002111155 02415
Gabriel Lima Guimaraes, Benjamin Sanchez- Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik. Objeetive-Reinforeed generative adversarial networks (ORGAN) for sequence generation models. May 2017.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, 2017.
10

---

Under review as a conference paper at ICLR 2020
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 1587-1596. JMLR. org, 2017.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:161 1.01
1.00 144, 2016.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. arXiv preprint arXiv: 04364, 2018.
Diederik P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling, Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems, pp. 3581-3589, 2014.
Peter Kirkpatrick and Clare Ellis. Chemical space. Nature, 432((719):823,, 2004.
Matt J Kusner, Brooks Paige, and José Miguel Hemándeez-Lobato. Grammar variational autoencoder. arXiv preprint urXiv:1703.0199 2017.
Greg Landrum. Rdkit: Open-source cheminformatics. URL http::/tpwwwwwo.c www.twt....k...... org
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander Gaunt, Constrained graph variational autoencoders for molecule design. In Advances in Neural Information Processing Systems 31, pp- 7806-7815. 2018.
Tengfei Ma. Jie Chen, and Cao Xiao. Constrained generation of semantically valid graphs via regularizing variational autoencoders. Advances in Neural Information Processing Systems 31, pp. 7113-7124. 2018.
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1:140022, 2014.
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1): 120-131, 2017.
N. Siddharth, Brooks Paige, Jan- Willem Van de Meent, Alban Desmaison, Noah Goodman, Pushmeet Kohli, Frank Wood, and Philip Torr. Learning disentangled representations with semi-supervised deep generative models. In Advances in Neural Information Processing Systems, pp. 5925-5935, 2017.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation small graphs using 2018.
variational autoencoders. arXiv preprint
Teague Sterling and John J Irwin. Zine 15-ligand discovery for everyone. Journal of chemical information and modeling, 55(11):2322 2031-2337770 2015.
Scott A Wildman and Gordon M Crippen. Prediction of physicochemical parameters by atomic contributions. Journal of chemical information and computer sciences, 39(5):668-877 1999.
Jiaxuan You, Bowen Liu, Rex Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy net- work for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems 3/, pp. 6412-6422, 2018.
11

---

Under review as a conference paper at ICLR 2020
6 APPENDIX
6.1 THE REGULARISER AND ITS RELATION TO THE MUTUAL INFORMATION MAXIMIZATION
The soft constrain in the loss 3 in fact, is equivalent to simple maximizing mutual information formulation between generated molecules x and the target property y provided to the generator. Assume the true conditional distribution is p(y/x):
[(y; x) Hy)) (y)*)
H() (xx-po(xxx) bty Ply p(y||))
(12)
We do not know the true distribution p(y|&), however, the RDKit provides an estimation p(y ()) of the distribution assuming a Gaussian distribution over error:
ply = (()
(13)
where f is the molecule property estimator, i.e., RDKit. We have:
Xyy: = (y)) - Ee (x) By 8e) [log p(y (x)
p(y' {x) p(y [x]]
-y) + Ex~po y) (P | (x) + Eyr~)(yls) log logp(y'(8) [x)]
2 Hyy + Ex~po (x yx By' log ((y
(14)
By following the Lemma 5.1 given in Chen et al. (2016), we have
I(y; 2 (y) Byw.ppohhse---ttttyyo/oooo//
(15)
As H (y) is constant, minimizing Ex- (x) y:l is equivalent maximizing [y; x) under the assumption that p(y(x) is close to p(y (x)
6.2 ARCHITECTURE AND TRAINING PROCEDURE DESCRIPTION
We use the same encoder and decoder network structure as Dai et al. (2018) with the only difference that our decoder takes as input the concatenation y, z. As GRU layers become computationally expensive when the sequences length increase, we also examined the model using less layer GRU, To be precise, the decoder in Dai et al. (2088) takes the from of dense hidden layer with ReLU activation followed by three layers GRU Chung et al. (2014). We tried two different settings of decoder: in the first setting, we feed the concatenation of y. z to dense layer then apply one layer GRU, in the second setting, to enhance the effect of in the decoder, we feed y not only to the dense layer but also to each layer GRU. Furthermore, we set the dimension of the latent representation to 56. For the oracle function estimator fw, we the same network architecture as the encoder (there is no parameter sharing) and we add one more fully connected layer followed by a Tanh transformation.
To speed up convergence, we initialize the fw from pre-training, where we train fw on the well- trained (maximum 500 epochs with early stopping) supervised VAE's decoder output to predict the molecules property value. We also initialize the parameters of the encoder/decoder networks with the partially trained supervised VAE model (after 40 epochs for QM9, 100 epochs for ZINC). We do not update and é, 0 simultaneously, instead we do alternate optimization. We update w continuously for five epochs while holding Φ 0 and do the same for updating Φ, O. We set the hyper-parameter value Aj to 50 and Ag to The mini-batch size is set to 300 for QM9 and 100 for ZINC. We use ADAM optimizer with learning rate 0.0001 and pytorch Ir-schedular on the validation loss. The general training algorithm is described in below algorithm blockT] In our experiment, to train fax we skipped the second term in step 7, which means we only train Far on the training data but not the newly generated molecules obtained by permuting the property. The reason for this is that, during the training, we found that it is easy for the model to learn to reconstruct but hard to conditionally generate the molecules with given properties while we have no guidance of what the molecules should look like. Further more, some combination of z and y are physically not feasible. In this case, when the conditional generation is not good enough yet during the training, we end up fitting fa on the miss represented molecules representations and it makes the optimization harder.
12

---

Under review as a conference paper at ICLR 2020
Algorithm 1 Training algorithm
1: Initialize po(x|z.y) y). 9o (z(z)x),
2: for 1=1,2..... N (maximum epoch number) do
3: for 1,2 : L,, sample minibatch D (X, Y) (x), of M samples do
4:
randomly permute the property set Y to obtain Y and define a label permuted mini batch D (X, Y") (:)
5:
0 = 0111 1 Vo (0.
1/4 Σ
yy (oll(( ysl19)
(x.Y))DD
6:
3 = 1- - 7( (0
7:
w/ w 7 Σ
Byforn Vulla x:1l +
Σ Bytdss) (xllf xxll) (x.30CD
(2.D))DD
6.3 USE A FIXED PRE-TRAINED PROPERTY PREDICTION FUNCTION
We also investigate the where train the property prediction function the well trained supervised VAE output and keep it fixed during the training of the main model. We give the performance tables 3 Using fixed fwr terms of reconstruction and generation performance, delivers mixed results However, in terms of conditional generation 4 it does perform better than the baselines but worse compared to the case where we also update fw during the learning.
QM9
ZINC
Model
Reconstruction
Valid%
Unique %
Novel Reconstruction %
Valid%
Unique
Novel %
CVAE
3.61
10.30
90.00
44.60
0.70
100
GVAE
96.00
60,20
80.90
53.70
7.20
100
SD-VAE
97.84
98.40
99.28
76.20
43.50
Sup- VAE-I-GRU
97.53
93.66
91.30
92.05
74.12
32.84
95.61
100
CGD-VAE-1 VAE-V-E---EEE 1-GUU
98.96
95.03
92.77
89.74
67.46
17.38
82.44
100
Sup GRU
CGD-VAB-3-GRU
96.9
93.8
98.29
89.55
81.80
37.78
98.70
100
Table 3: Sup-VAE- VAE-1-GRU /Sup Sup-VAAB3-GRR- supervised version of SD-VAF model where is been
feed to only the first layer/all layer decoder. CGD-VAE: our model, conditional generation with disentangling. The result for CVAE and GVAE are taken from the literature, "_" refers that those measures are not reported. The rest of baseline result obtained by rerunning (SD-VAE) and editing the original code
Model
2 & (z)
& ~ g;2(x)
Sup- VAE-I-GRU
0.5420
0,2526
CGD-VAE-1-GRU
0.6331
0,3835
QM9
Sup Sup-VAE-3..m VAE-3-GRU
0,6958
0.4204
0.6665
0,4507
Sup-VAre -A----A---GU
0.2301
0.0481
BED-VAE-1-0
0.2981
0.0866
ZINC
Sup- VAE-3-GRU
0.3638
0.1818
CGD-VAE-3-GRU
0.3765
0.1310
Table 4: Correlation between the desired input property and the obtained property when is sampled from the approximate learned prior, z ~ qo (z), (conditional generation), and when is sampled from the learned posterior given some (z x), (property transfer).
13

---

Under review as a conference paper at ICLR 2020
6.4 VIRTUAL SCREENING
The figure 8 displays the results of a virtual screening method, where we select from the full dataset five molecules which are structurally similar X in figure in section and have logP values close to B- As we can see the molecule that our model generates is new one.
HO
-0.588
-C.668
-0.568
-0.555
-0.555
Figure 8: Molecules selected with virtual screening over the full dataset in manner that they structurally similar to the A molecule of figure while they have a logP value which is close to the logP value of the B molecule also of figure 4
6.5 METRICS FUNCTION OF
The validity and novelty were mainly used to assess purely the generative model performance. However, also interesting see the model capable generating valid and novel molecules we start from an existing molecules and drift away from the original label, i.e., if we get from q(]xx), and then compare different values from p(x(z, y') as y' moves far from y. We randomly sample molecule x whose LogP is from the test set, then sample 10 z from (([[x). For each such z we couple it with a y' that is different that y, and sample 10 molecules from p(x|z, y'). Eventually, for each such y', starting from original molecule we report the validity, uniqueness and novelty as a function y y'. The figure disslaas result of repeating above process for 20 randomly sampled (x. y) along 100 grid points for y yy. The result confirms that on big data set, conditional generative models uniqueness, validity and novelty performance is not affected by the size of the modification done the property. However, small dataset, uniqueness is not affected. As expected, novelty increases as the properties modification size increases and validity drops slightly as the property modification size increase.
generated 100 molecules, and
Generation performance peoperty modification
Validity. Novelty, Uniqueness changes y
10
10
Percentage
Percentage
06
Validity Novelty Uniqueness
0.6
Novelty Uniqueness
0.4
0.4
y-y'
y-y"
Figure 9: CGD-VAE-3-GUIE model validity, nov- clty, uniqueness performance property transfer task as function of y y' on QM9 dataset.
Figure 10: CGD-VAE-3-GRU model validity,
novelty, uniqueness performance on property transfer task as a function of y on ZINC dataset.
6.6 SAMPLING FROM APPROXIMATED MARGINAL POSTERIOR FOR GENERATION
With our model, during the generation, we observe that sampling from the approximated marginal posterior improves generation performance when compered sampling from the prior. Here
14

---

Under review as a conference paper at ICLR 2020
investigate if this findings holds for other baseline models not. We explored the behavior of baselines when the z is sampled from the approximate marginal posterior, we observe that for CVAE and GVAE the validity did not change (as the ((z) and p(z) are essentially identical); for the SD-VAE the validity increases (Table
SDVAE
Valid%
Unique %
Novel %
2 do (2)
56.05
80.69
100
p(z)
43.50
82.26
100
Table 5: Baseline model performance on ZINC
6.7 MOLECULE PROPERTY OPTIMIZATION
Figure is the visualization the generated molecules from property optimization task, given in
figure section 4 The molecules are generated by increasing the logP of given molecule, i.e. hold z fixed and increase the y value. Among the all generated molecules, 19 of them are unique.
Figure 11: Molecules generated with increasing logP
15


Reviews:
[{"review_id": "HJlbqymTYH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors introduce a variational autoencoder for conditional generation of molecules. The model is borrowed from text-based style transfer, applied here on sequence (SMILES) representation of molecules rather than viewing molecules as graphs (as more recent approaches). From a modeling point of view, the main new part is an additional regularizer whose role is to 1) ensure that the property used as input during generation matches the property derived from the generated molecule, and 2) to dissociate the latent molecule representation in the autoencoder (loosely speaking, its overall structure) from the property being controlled. This regularizer is just a squared difference between predicted and actual properties, averaged over independent samples from latent states and properties which are parametrically mapped to predicted properties via the decoder state (so as to be able to backprop).\n\nThe resulting ELBO criterion for the VAE, together with the regularizer, could be in principle directly used to train all the model parameters (encoder, decoder, property mapping) but apparently this criterion does not result in a \"stable\" estimation procedure. The model is then amended in various ways. \n\nThe first change consists of introducing a property prediction part directly into ELBO so that the posterior over latent states will also be affected by property prediction (previously property prediction would only appear in the additional regularizer). Beyond stability that the authors cite as the reason, this is likely also needed because in eq (2) the posterior over latent states given the molecule does not depend on the property. The posterior is derived from a v-structure where latent states and properties appear as independent parents of the molecule. Since for the training cases the property is fixed, this would be fine with an arbitrarily complex posterior encoding. However, when the encoder is defined parametrically, this dependence may be needed, and may relate to the stability issues. It seems a bit roundabout way of putting the dependence back in, if so. The authors should comment on this further. \n\nThe other simplification that is introduced is that the prior over the latent states in the regularizer is modified to be the average posterior of training encodings, and later further collapsed (for computational reasons) to an adaptively estimated simple Gaussian with variance that matches the average posterior variance.\n\nIt's a bit unsatisfying that the authors only use and experiment with a simple logP as the property to control in the model. This is not really a particularly relevant property to control. The approach should in principle generalize to other properties but the many adjustments needed to make it work makes this a little questionable. Also, the results are a bit limited (only logP) and not encouraging (see comments below). \n\n- how is validity defined? SMILES syntax or validity as a molecule? \n\n- is table 1 reconstruction really autoencoder reproduction percentage? Is the correct property for the molecule being reconstructed offered as an additional input to the authors' model? isn't this a bit unfair?\n\n- the correlation between property given as input and the property derived from the generated molecule does not seem like a good metric. Wouldn't a model that overfits (essentially spitting out the nearest property neighbor from the training set) do particularly well according to this metric?\n\n- it seems Figure 6 can be interpreted as also indicating some degree of overfitting?\n\n- the stacked LSTM molecular generator baseline that takes the property as input seems to do much better in terms of maintaining the property in its generated molecules and it also realizes a more diverse collection of molecules. It's true that it doesn't support incremental molecular manipulation directly, eg, if one is keen on keeping the modified molecule close to a starting point. But it is so much better at maintaining the input property in its diverse realizations that perhaps one should instead invest in a tailored sampling procedure to obtain realizations close to a chosen reference.\n\nThe literature references seem to omit all molecular generation work since 2018 \n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HJeJEM-0tS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# Summary\nThe paper considers the problem of generating molecules with desired properties using a variant of supervised variational auto-encoders. The key novelty is the idea to learn a representation with disentangled molecular properties which can be modified during generation of novel molecules.\n\nTo account for the diversity of molecules in a particular target class the conditional probability of a molecule x given a target property y is modelled with a latent variable z, i.e., p(x | y) = \\int p(x | z, y) p(z) dz.\n\nThe \"style transfer\" is achieved by taking an initial molecule x and computing the posterior of novel molecule x' with modified property y' via p(x' | y', x) = \\int p(x' | y', z) p(z | x) dz.\n\nThe latent representation is learned using a supervised auto-encoder (Kingma et al., 2014), formally specified in Eq. (2). The optimization problem is then further extended by imposing a soft constraint that the molecules from a particular class exhibit a certain property (Eq. 3). The reason for this constraint is to enforce that the conditional probability p(x | y, z) takes into account the information in label y. This part might not be sufficiently well-motivated and would benefit from strengthening the argument for the property predictor and soft constraint (would an illustration be possible here?). In particular, why should it not be possible to encode (and keep fixed during training) the information present in y into the sufficient statistics of p(x | y, z)?\n\nThe soft constraint involves an oracle function (see Eq. 3), which evaluates the designed molecules. Section 3.2 deals with the approximation of the oracle via a property predictor function to side-step the fact that the oracle is not necessarily differentiable or CPU-bound. Section 3.3 and 3.4 deal with joint optimization of the supervised variational auto-encoder and gradient estimates.\n\nThe approach is evaluated on two datasets, relative to state-of-the-art baselines based on variational auto-encoders and deep learning. The main goals of the experiments are to establish that the approach can learn useful disentangled conditional distributions over molecules and to assess the extent of improvement in the data generation process as a result of using the soft constraint term. The focus of the experiment is on generating molecules with certain range of logP values. The data generating process is simulated/evaluated while controlling for the logP value range and molecular structure.\nThe experiments provide an objective assessment of the merits of the approach and indicate clear advantages over prior work. \n\n\n# Recommendation\nThe paper is very well written, easy to follow, and properly structured. The work is novel and the idea to disentangle molecular properties from the target property is quite interesting. The experiments are detailed and compare to baselines based on variational auto-encoders and deep learning. The part that could be improved is the choice of the target property. In particular, it would be great to evaluate the approach on the problem of finding molecules binding to a certain protein site. Such a problem is likely to exhibit scaffold hops and activity cliffs which make drug design problems very difficult (e.g., see [6-8]).\n\n\n# Related work (additional references)\n- While mapping of the discrete space of molecules to a continuous latent space (in active learning approaches combined with VAE) allows for gradient computation and the use of active learning/optimization techniques, the very difficult problem of finding latent space pre-images persists (in general, the problem should be in the NP class). The latter refers to finding a molecule that maps to a fixed point in the latent space. There have, however, been some efficient approximations in special cases [1-3].\n- In [4] and [5], an approach for generation of molecules with desired properties has been pursued with posterior sampler based on a conditional exponential family model. The representation is based on a tuple kernel mapping mapping (x, y) to some reproducing kernel Hilbert space (without disentanglement). The approach provides a consistent data generation process and a mean to deal with the fact that designs are not independent and identically distributed (e.g., Eq. 2 assumes that examples are IID).\n\n\n# Black-box target property\n- The logP property might not be a good proxy for the effectiveness in generating molecules with desired binding activities (to some protein site) because (to the best of my knowledge) it does not exhibit the \"activity cliff property\" characteristic to drug design. This refers to the property that a small change in molecular structure results in a completely different activity level (e.g., see [6-8] and [4]). In this sense, the \"style transfer\" might be beneficial for finding molecules binding to a certain protein site that are structurally similar to some available molecules with bad activity levels (for which a synthesis path might be known or easy to derive).\n- An actual black-box property (realized via a docking program) that is expensive to evaluate served as a target property in [5]. That docking program (supplied with suitable binding/docking constraints) can provide a nice proxy for the actual generation of molecules.\n\n\n# References\n[1] G. H. Bakir, J. Weston, and B. Schoelkopf. Learning to find pre-images, NIPS 2004.\n[2] C. Cortes, M. Mohri, and J. Weston. A general regression technique for learning transductions, ICML 2005.\n[3] S. Giguere, A. Rolland, F. Laviolette, and M. Marchand. On the string kernel pre-image problem with applications in drug discovery, ICML 2015.\n[4] D. Oglic, R. Garnett, and T. Gaertner. Active search in intensionally specified structured spaces, AAAI 2017.\n[5] D. Oglic, S. Oatley, S. Macdonald, T. Mcinally, R. Garnett, J. Hirst, and T. Gaertner. Active search for computer-aided drug design, Molecular Informatics 2018.\n[6] G. Schneider and U. Fechner. Computer-based de novo design of drug-like molecules, Nature Reviews Drug Discovery, 2005.\n[7] P. Schneider and G. Schneider. De novo design at the edge of chaos. Journal of Medicinal Chemistry, 2016.\n[8] J. Scannell, A. Blanckley, H. Boldon, and B. Warrington. Diagnosing the decline in pharmaceutical R&D efficiency. Nature Reviews Drug Discovery, 2012.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Byo_bZRtB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a VAE-based conditional molecular graph generation model. For that purpose, the disentanglement approach is adopted in this paper: learn to separate property information from the structure representation of a molecular graph. The authors use the supervised VAE objective since the KL regularizer in the objective has reportedly disentanglement-promoting effect. The final objective function is a standard VAE ELBO plus a penalty term of the property value prediction error. Non-differentiable property estimation is conducted via stochastic sampling expectation with the help of an external program (RDKit).  \n\nThe proposed model directly optimizes the generation model conditioned on the property values in a single objective function. This is preferable compared to many existing molecular graph generations, where property optimization is often carried out after the core generative models are trained and fixed. \nDerivation of a stable lowerbound (Eq. 11) is another plus. \n\nMy main concern about the paper is a weak survey for disentanglement researches. Since the core of graph generation model is not original of the authors (adopted from Dai+, 2018), the main technical advancement of the paper should be related to the disentangling VAE modeling. \nCurrent researches in disentangling VAEs go beyond the InfoGAN and beta-VAE. I present a part of the must-referred papers below:\n\n[Gao19] Gao+, “Auto-Encoding Total Correlation Explanation”, AISTATS, 2019. \n[Kim_Mnih18] Kim and Mnih, “Disentangling by Factorising”, ICML, 2018.  \n[Ryu19] Ryu+, “Wyner VAE: Joint and Conditional Generation with Succinct Common Representation Learning”, arxiv:1905.10945, 2019. \n\nAmong them, [Ryu19] is closely related to the proposed framework. In my understanding, the variable dependency structure studied in this paper (Fig. 1) is studied by [Ryu19]. The authors should clearly state the novelty of the proposed work in the literature of these disentanglement VAE works. \n\nI think the L_disent (Eq.5) is not a penalty for disentanglement: it enforces the model to correctly predict target property values, and do not say anything for factor disentanglement, correct? \n\nI have a few concerns about the experiment designs. \nIn the table 1, reconstruction performance evaluations, the authors chose string(SMILE)-based molecular graph generation models. However, it is largely admitted in the molecular graph generation studies that the graph-based generative models generally performed better. I want justifications for the choice of string-based generation models. Extending the table 1 with graph?base SotA generation models will strengthen the manuscript greatly. \nFor example:\n\n[Jin18] Jin+, “Junction Tree Variational Autoencoder for Molecular Graph Generation”, ICML, 2018. \n[You18] You+, “Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation”, NeurIPS, 2018. \n\nEspecially, [You18] directly optimize the property values of the generated graph. This is closely related to the goal of this paper, thus a proper reference and discussions are strongly expected. \n\n\nFigure 5. is an attractive visualization of the latent (z, y) vectors. However, the authors do not provide how to understand the figure. I GUESS that the authors want to show that learned z and y are somehow disentangled: the structural changes of molecular graphs are less sensitive to the vertical axis (property value) than the horizontal column (different). Please clearly state key messages of each figure and table. \n\nThere are several presentation issues. They are details, but fairly degrades the readability of the manuscript.  \n- Too small letters (alphabets) in Figure 3, 4, 6, it is simply unreadable so I cannot tell main messages of these figures (So I do not evaluate these figures positively). \n- In table 1, what are the significance figures of the presented scores? Some scores have 3 digits, others have 4 digits. Some are rounded at 0.01 precision but others are round at 0.1 precision. \n\nEvaluation summaries\n+ A graph generation model combining conditional property optimization in a single objective function.\n+ A new lowerbound for stable training (Eq.11)\n+ The property and the structure of the graph are somehow disentangled in the experiment (Fig. 5). \n+- Disentanglement effect is not modeled directly in the proposal. L_disent is not for disentanglement but for property regression.\n-- Survey for disentangling VAEs is not enough. This makes the proposal less convincing in terms of the novelty and the contribution compared to the recent dissent-VAEs. \n- Not compared with SotA graph-based molecular graph generation models. \n- Key messages of figures and tables are not clearly stated (e.g. Fig.5). \n-- Some figures are simply unreadable. The significance figure of the table 1 is unclear. These are formatting details but essential for readability. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper aims to generate molecules with desired properties using a variant of supervised variational auto-encoders. Disentanglement is encouraged among the style factors.  The reviewers point out that the idea is nice, but authors avoid quantitative comparison with SotA Graph-based generative models.  Especially, the JT-VAE is acknowledged as a strong baseline widely in the community and is a VAE-based model, it is important to do these comparisons. 
Decision: Reject