All SMILES Variational Autoencoder for Molecular Property Prediction and Optimization
Authors: Zaccary Alperstein
Artem Cherkasov
Jason Rolfe
Venue:  (2020)
Keywords: generative modelling
variational autoencoder
chemistry
cheminformatics
chemoinformatics
molecular property optimization

Abstract: Variational autoencoders (VAEs) defined over SMILES string and graph-based representations of molecules promise to improve the optimization of molecular properties, thereby revolutionizing the pharmaceuticals and materials industries. However, these VAEs are hindered by the non-unique nature of SMILES strings and the computational cost of graph convolutions. To efficiently pass messages along all paths through the molecular graph, we encode multiple SMILES strings of a single molecule using a set of stacked recurrent neural networks, harmonizing hidden representations of each atom between SMILES representations, and use attentional pooling to build a final fixed-length latent representation. By then decoding to a disjoint set of SMILES strings of the molecule, our All SMILES VAE learns an almost bijective mapping between molecules and latent representations near the high-probability-mass subspace of the prior. Our SMILES-derived but molecule-based latent representations significantly surpass the state-of-the-art in a variety of fully- and semi-supervised property regression and molecular property optimization tasks.

Under review as a conference paper at ICLR 2020
ALL SMILES VARIATIONAL AUTOENCODER FOR MOLECULAR PROPERTY PREDICTION AND OPTIMIZATION
Anonymous authors Paper under double-blie blind review
ABSTRACT
Variational autoencoders (VAEs) defined over SMILES string and graph-based representations of molecules promise to improve the optimization molecular properties, thereby revolutionizing the pharmaceuticals and materials industries. However, these VAEs are hindered by the non-unique nature of SMILES strings and the computational cost of graph convolutions. To efficiently pass messages along all paths through the molecular graph, we encode multiple SMILES strings of a single molecule using set of stacked recurrent neural networks, harmonizing hidden representations of each atom between SMILES representations, and use attentional pooling to build final fixed-length latent representation. By then decoding to a disjoint set of SMILES strings of the molecule, our All SMILES VAE learns an almost bijective mapping between molecules and latent representations near the high- tettt-roobbbtt--ooane mass subspace of the prior. Our based latent representations significantly surpass the state-of-the-an in a variety of fully- and semi-superviseee property regression and molecular property optimization tasks.
but molecule-
INTRODUCTION
The design of new pharmaceuticals, OLED materials, and photovoltaics all require optimization within the space of molecules (Pyzer-Knapp et al., 2015). While well-known algorithms ranging from gradient descent to the simplex method facilitate efficient optimization, they generally assume a continuous search space and a smooth objective function. In contrast, the space of molecules is discrete and sparse. Molecules correspond to graphs, with each node labeled by one of ninety-eight naturally occurring atoms, and each edge labeled as single, double, or triple bond. Even within this discrete space, almost all possible combinations of atoms and bonds do not form chemically stable molecules, and so must be excluded from the optimization domain, yet there remain as many as 10 small molecules to consider (Bohacek et al.. 1996). Moreover, properties of interest are often sensitive to even small changes to the molecule (Stumpfe & Bajorath, 2012). so their optimization is intrinsically difficult.
Efficient, pradient-based optimization can be performed over the space of molecules given a map between continuous space, such I. the n-sphere, and the space of molecules and their properties (Sanchez-Lengeling & Aspura-Guzik, 2018). Initial approaches of this form trained variational autoencoder (VAE) (Kingma & Welling. 2013; Rezende al., 2014) on SMILES string representations of molecules (Weininger, 1988) to learn a decoder mapping from a Gaussian prior to the space of SMILES strings (Gómez-Bombarelli et al., 2018). A sparse Gaussian process on molecular properties then facilitates Bayesian optimization of molecular properties within the latent space (Dai et al., 2018; K-mez-Bombaree et al., 2018; Kusner et al., 2017; Samanta al., 201 8), or neural network regressor from the latent space to molecular properties can be used to perform gradient descent on molecular properties with respect to the latent space (Aumentado- Armssddod.......nggnn..o 2018; Jin et al., 2018; Liu et al., 2018; Mueller et al., 2017). Alternatively, semi-supervised VAEs condition the decoder on the molecular properties (Kang & Cho, 2018; Lim et al., 2018), so the desired properties can be specified directly. Recurrent neural networks have also been trained to model SMILES strings directly, and tuned with transfer learning, without an explicit latent space or encoder (Gupta al., 2018; Segler al., 2017).

---

Under review as a conference paper at ICLR 2020
SMILES, the simplified molecular-inple input line-entry system, defines a character string representation of a molecule by performing a depth-first pre-order traversal of a spanning tree of the molecular graph, emitting characters for each atom, bond, tree-traversal decision, and broken cycle (Weininger, 1988). The resulting character string corresponds to a flattening of a spanning tree of the molecular graph, shown in Figure 1. The SMILES grammar is restrictive, and most strings over the appropriate character set do not correspond to well-defined molecules. Rather than require the VAE decoder to explicidy learn this grammar, context-free grammars (Kusner et al., 2017), and attribute grammars (Dai et al., 2018) have been used to constrain the decoder. increasing the percentage of valid SMILES strings produced by the generative model. Invalid SMILES strings and violations of simple chemical rules can be avoided entirely by operating on the space of molecular graphs, either directly (De Cao & Kipf, 2018; Ma et al., 2018; Li et al., 2018; Liu et al., 2018; Simonovsky & Komodakis, 2018) 8) or via junction trees (Jin et al., 2018).
Every molecule is represented by many well-formed SMILES strings, corresponding to all depth first traversals of every spanning tree of the molecular graph. The distance between different SMILES strings of the same molecule can be much greater than that between SMILES strings from radically dissimilar molecules (Jin et al., 2018), shown in Figure 8 of Appendix A. A generative model of in- dividual SMILES strings will tend to reflect this geometry, complicating the mapping from latent space to molecular properties and creating unnecessary local optima for property optimization (Vinyals et al., 20. To address this difficulty, sequence-to-seen -o-sequence transcoders (Sutskever et al., 2014) have been trained map between different SMILES strings of single molecule (Bjerrum, 2017; Bjerrum & Sattarov, 2018; Winter et al., 2019b;a).
Reinforcement learning, often combined with adversarial methods, has been used train progressive molecule growth strategies (Guimaraes et al., 2017: Jaques et al., 2017: Olivecrona et al., 2017; Putin et al., 2018; You al.. 2018; Zhou et al., 2018). While these approaches have achieved state-of-the-ee optimization of simple molecular properties that can be evaluated quickly in silico, critic-free techniques generally depend upon property values of algorithm-generee
molecules (but see (De Cao & Kipf, 2016 8; Popova et al., 2018)), and so scale poorly to real-world properties requiring coneuconsuinng wet-lab experiments.
OH
NH2
N
Figure I: The molecular graph of the amino acid Tryptophan (a). To construct a SMILES string, all cycles are broken, forming a spanning tree (b); a depth-first traversal is selected (c); and this traversal is flattened (d). The beginning and end of intermediate branches in the traversal are denoted by ( and respective. The ends of broken cycles are indicated with matching digits. The full grammar is listed in Appendix D. A small set of SMILES strings can cover all paths through a molecule (c).
Molecular property optimization would benefit from a generative model that directly captures the geometry of the space of molecular graphs, rather than SMILES strings, but efficiently infers a latent representation sensitive to spatially distributed molecular features. To this end, we introduce the All SMILES VAE, which uses recurrent neural networks (RNNs) on multiple SMILES strings to implicitly perform efficient message passing along and amongst many flattened spanning trees of the molecular graph in parallel. A fixed-length latent representation is distilled from the variable- length RNN output using attentional mechanisms. From this latent representation, the decoder RNN reconstructs a set of SMILES strings disjoint from those input to the encoder, ensuring that the latent representation only captures features of the molecule, rather than its SMILES realization. Simple property regressors jointly trained on this latent representation surpass the state-of-the-art for molecular property prediction, and facilitate exceptional gradient-based molecular property optimization when constrained to the region of the prior containing almost all the probability around it. We further demonstrate that the latent representation forms near- bajr-biecction with the space of molecules, and is smooth with respect to molecular properties, facilitating effective optimization. For complete delineation of our novel contributions relative to past work, Appendix B.4.
2

---

Under review as a conference paper at ICLR 2020
2 EFFICIENT MOLECULAR ENCODING WITH MULTIPLE SMILES STRINGS
A variational autoencoder (VAE) defines a generative model over an observed space I in terms of a prior distribution over latent space p(z) and a conditional likelihood of observed states given the latent configuration p(x 2) (Kingma & Welling, 2013; Rezende et al., 2014). The true log- likelihood log [r(x)] = log [ p(=)p((() is intractable, so the evidence lower bound (ELBO), based upon a variational approximation g( ((2lz) to the posterior distribution, is maximized instead: C= (aja [log p(>]==]] DRL [[===z]][p(<{)) The ELBO implicitly defines stochastic autoencoder, with encoder (z)(xr) and decoder p(x[==)
Many effective molecule encoders rely upon graph convolutions: local message passing in the molecular graph, between either adjacent nodes or adjacent edges (Duvenaud et al., 2015; Kearnes ct al., 2016; Kipf & Welling, 2016a; Li et al., 2015; Lusei et al., 2013). To maintain permutation symmetry, the signal into each node is a sum of messages from the adjacent nodes, but may be a function of edge type, or attentional mechanisms dependent upon the source and destination nodes (Ryu et al., 2018). This sum of messages is then subject to linear transformation and pointwise nonlinearity. Messages are sometimes subject to gating (Li et al., 2015), like in long short- term memories (LSTM) (Hochreiter & Schmidhuber, 1997) and gated recurrent units (GRU) (Cho et al., 2014), as detailed in Appendix B.
traditional convolutional neural network applied to images (Krizhevsky et al., 2012 LeCun et al., 1990), with constant-resolution hidden layers (He et al., 2016) and two kernels: a 3 x 3 average-pooling kernel that sums messages from adjacent pixels (corresponding to adjacent nodes in molecular graph), and a trainable 1 x 1 kernel that transforms the message from each pixel (node) independently, before a pointwise nonlinearity. While such convolutional networks are now standard in the visual domain, they use hundreds of layers to pass information throughout the image and achieve effective receptive fields that span the entire input (Szegedy et al., 2016). In contrast, molecule encoders generally use between three and seven rounds of message passing (Duvenaud et al., 2015; Gilmer et al., 2017; Jin et al., 2018; Kearnes et al., 2016; Liu et al., 2018; Samanta al., 2018; You et al., 2018). This limits the computational cost, since each iteration of message passing only propagates information a geodesic distance of one within the molecular graph.' In the case of the commonly used dataset of 250,000 drug-like molecules Kommez-Bombarelle et al., 2018), information cannot traverse these graphs effectively, as their average diameter is 11.1, and their maximum diameter is 24, as shown in Appendix A.
Message passing on molecular graphs is analogous
Non-local molecular properties, requiring long-range information propagation along the molecular graph, are of practical interest in domains including pharmaceuticals, photovoltaics, and OLEDs. The pharmacological efficacy of molecule generally depends upon high binding affinity for a particular receptor or other target, and low binding affinity for other possible targets. These binding affinities are determined by the maximum achievable alignment between the molecule's 's electromagnetic fields and those of the receptor. Changes to the shape or charge distribution in one part of the molecule affect the position and orientation at which it fits best with the receptor, inducing shifts and rotations that alter the binding of other parts of the molecule, and changing the binding affinity (Clayden et al., 2001). Similarly, efficient next- gext-genneation OLEDs depend on properties, such as the singlet- energy gap, that are directly proportional to the strength of long-range electronic interactions across the molecule (Im et al., 2017). The latent representation of VAE can directly capture these local, nonlinear properties only if the encoder passes information efficiently across the entire molecular graph.
Analogous to graph convolutions, gated RNNs defined directly on SMILES strings effectively pass messages, via the hidden state, through a flattened spanning tree of the molecular graph (see Figure 1). The message at each symbol in the string is weighted sum of the previous message and the current input, followed by a pointwise nonlinearity and subject to gating. as reviewed in Appendix B. This differs from explicit graph-based message passing in that the molecular graph is flattened into a chain corresponding to a depth-fire first pre-order traversal of spanning tree, and the set of adjacent nodes that affect a message only includes the preceding node in this chain. Rather than updating all messages in weeeeeeiieiii parallel, RNNs on SMILES strings move sequentially down the chain, so earlier messages influence
All-to-all connections allow fast information transfer, but computation is quadratic in graph size (Gilmer al., 2017; Kearnes et al., 2016). Lusci et al. (2013) considered set of DAGs rooted at every atom, with full message propagation in single pass.
3

---

Under review as a conference paper at ICLR 2020
all later messages, and information propagate through all branches of flattening of spanning tree in a single pass. With a well-chosen spanning tree, information can pass the entire width of the molecular graph in single RNN update. The relationship between RNNs on SMILES strings and graph-based architectures is further explored in Appendix B.
3 MODEL ARCHITECTURE
To marry the latent space geometry induced by graph convolutions to the information propagation efficiency RNNs on SMILES strings, the All SMILES encoder combines these architectures. takes multiple distinct SMILES strings of the same molecule as input, and applies RNNs to them parallel. This implicitly realizes a representative set of message passing pathways through the molecular graph, corresponding the depth-fire frrph--fistpp pre-order traversals of the spanning trees underlying the SMILES strings. Between each layer of RNNs, the encoder harmonizes homologous messages between parallel representations of the multiple SMILES strings. In this harmonization, all messages to single atom across the multiple SMILES strings are replaced with their pooled average, so that information flows along the union of the implicit SMILES pathways.
Initially, the characters of the multiple SMILES strings are linearly embedded, and each string is preprocessed by BiGRU (Cho al., 2014), followed by linear transformation, produce the layer representation H for each SMILES string i. For each SMILES string and layer H is sequence of vector embeddings, one for each character of the original SMILES string, collectively forming a matrix. The encoder then applies a stack of modules, each of which harmonizes atom representations across SMILES strings, followed by layer norm, concatenation with the linearly embedded SMILES input, and GRU applied to the parallel representations independently, as shown Figures 2 and 3.
Linear embedding
cle(Ci)ene
II:
Layer norm
GRU
11
Harmonize atoms
eInce(Cl)el
Layer norm
GRU
Linear embedding
Figure 2: In each layer of the encoder after the initial BiGRU and linear transformation, hidden states corresponding to each atom are harmonized across encodings of different SMILES strings for common molecule, followed by layer norm and a GRU on each SMILES encoding independently.
C
CI
(a) Original molecule
(b) Harmonization two SMILES strings representing same molecule
Figure 3: To pass information between multiple SMILES representations of molecule, the encoder harmonizes the representation of each atom. Homologous messages corresponding to the same atom pooled (b), and the original messages replaced with this pooled message, reversing the information flow of (b).
Multiple SMILES strings representing a single molecule need not have the same length, and syntactic characters indicating branching and ring closures rather than atoms and bonds do not generally match. However, the set of atoms always consistent, and bijection can be defined between homologous atom characters. At the beginning of each encoder module (Figure 2), the parallel
4

---

Under review as a conference paper at ICLR 2020
inputs corresponding single, common atom of the original molecule are pooled, as shown in Figure 3. This harmonized atom representation replaces the original in each the input streams for the subsequent layer normalizations and GRUs, reversing the information flow of Figure 3. To realize atom harmonization, we experimented with average and max pooling, but found element- wise sigmoid gating to be most effective (Dauphin al., 2017; Li et al., 2015; Ryu et al., 2018): a' = + Ex (ax O a (W [kk, R >> ax] + 6)), where [x, y] is the concatenation of vectors x and y. and the logistic function (x) is applied element- wise. The pooling effectively sums messages propagated from many adjacent nodes in the molecular graph, analogous to graph convolution, but the GRUs efficiently transfer information through many edges in each layer, rather than just one. The hidden representations associated with non-atom, syntactic input characters, such as parentheses and digits, are left unchanged by the harmonization operation.
The approximating posterior distills the resulting variable-Iength encodings into fixed-length hierarchy of autoregressive Gaussian distributions (Rolfe, 2016). The mean and log-variance of the first layer of the approximating posterior, 21. is parametrized by max-pooling the terminal hidden states of the final encoder GRUs, followed by batch renormalization (Ioffe, 2017) and a linear transformation, as shown in Figure 4. Succeeding hierarchical layers use Bahdanau-style attention (Bahdanau et al., 2014, reviewed in Appendix B.2) over the pooled final atom vectors, with the query vector defined by one-hidden-layer network of rectified linear units (Rel.Us) given the concatenation of the previous latent layers input. This is analogous to the order-invariant encoding of set2set, but an output is produced at each step, and processing is not gated (Vinyals al., 2015). The attentional mechanism is also effectively available to property regressors that take the fixed-length latent representation as input, allowing them to aggregate contributions from across the molecule. The output of the attentional mechanism is subject to batch renormalization and linear transformation to compute the conditional mean and log-variance of the layer. The prior has similar autoregressive structure. but uses neural networks of ReLUs in place of Bahdanau-style attention, as there are no atom vectors. This is illustrated in Appendix B in Figure 11, briefly, each layer of the hicrarchy is a two layer neural network conditioned on the previous latent samples, that is, Po (2i, 2x) p(21) T Po, (zi(=<s). For molecular optimization tasks, we usually scale up the KL-termm in the ELBO by the number of SMILES strings the decoder, analogous multiple single-SMILES VAEs in parallel; we leave this KL term unscaled for property prediction.
Pool atoms
HI
GRU
k
Pool
Lin ce
Z
NN
Att
Z2
NN
Att
Z3
μ.
q
H
GRU
Figure The approximating posterior is an autoregressive set of Gaussian distributions. The mean (p) and log-variance (log o²) of the first subset of latent variables z is linear transformation of the max-pooled final hidden state of GRUs fed the encoder outputs. Succeeding subsets z, are produced via Bahdanau-style attention with the pooled atom outputs of the GRUs as keys (k), and the query (ạ) computed by a neural network on Zzi.
The decoder is a single-layer LSTM, for which the initial cell state computed from the latent representation z = [21, 22, ... ] by a neural network, and a linear transformation of the latent representation concatenated onto each input. It trained with teacher foreing to reconstruct set of SMILES strings disjoint from those provided to the encoder, but representing the same molecule. As conventional language models, the decoder LSTM autoregressively produces sequence of categorical distributions for each successive SMILES character conditioned on the preceding characters. We decode multiple SMILES in parallel with shared decoder parameters, conditioned on the same latent representation. When decoding any SMILES from the same molecule, the decoder may predict characters with low confidence, but as the decoding proceeds, the decoder is conditioned on previous characters of the SMILES string, becoming more certain about which string is meant to be decoded. Grammatical constraints (Dai et al., 2018; Kusner et al., 2017) can naturally be enforced by parsing the unfolding character sequence with pushdown automaton, and constraining the final
5

---

Under review as a conference paper at ICLR 2020
softmax of the LSTM output at each time step to grammatically valid symbols. This is detailed in Appendix D, although we leave the exploration of this technique to future work.
The All SMILES VAE is generative model over both the structure and properties of molecules M, NN so we define the conditional likelihood to be p(M[2) = P (pM]z) II, P (2)M)z) where {at) J=1 a set of N SMILES strings of a molecule M with properties prM Unlike a conventional VAE, the representation of the molecule M input to the encoder g(=[M) is not identical to the target of the conditional likelihood p(M[z); rather, it comprises a set of SMILES strings (a:") as of the molecule M disjoint from the decoding target, and does not include the molecular properties. Nevertheless, both encoder input and decoder target correspond to a single molecule M. The conditional log likelihood of the molecular properties log p (p~M|z) is implicitly parametrized by scaling its contribution to the ELBO by X. For instance, if p (pM]=) is unit-variance Gaussian distribution, then À sets the effective variance to A- Finally, when optimizing molecular properties, scale the KL term by M, the number of SMILES strings in the decoder, rendering the ELBO analogous to multiple single- VAEs in parallel. The resulting ELBO is:
M
L EE E2- (=)(()(( A logg(((22) + Σ
log 1ogp(1z12)
+ +M- DKL [9 (=l(<>)) [l)o
In our experiments we parametrize the regressors and classifiers used in p(plz) by single linear layer, with an activation function to put the predicted property into its correct range. Since the SMILES inputs to the encoder are different from the targets of the decoder, the decoder is effectively trained to assign high probability to all SMILES strings of the encoded molecule. The latent representation must capture the molecule as a whole, rather than any particular SMILES input to the encoder. To accommodate this intentionally difficult reconstruction task, facilitate the construction of a bijection between latent space and molecules, and following prior work (Kang& & Cho, 2018; Winter et al., 2019a), we use a width-5 beam search decoder to map from the latent representation to the space of molecules at test-time. In all experiments, for encoding, and N = 5 disjoint SMILES strings as the decoding target. Further architectural details presented in Appendix B.
use set of M = 5 randomly selected SMILES strings
3.1 COMPUTATIONAL COMPLEXITY
Since the length of a SMILES string is linear in the total number of bonds b, the computational complexity of each layer of the All SMILES encoder O(M -)), where M is the number of random SMILES strings of the molecule. Similarly, the complexity of each layer of graph convolution (b). However, to pass information through the entire molecule, graph convolutions require a number of layers proportional to the graph diameter. Molecular graph convolutions generally use fixed architecture for all molecules. In principle, the maximum diameter of molecule is equal to the number of bonds. As a result, the computational complexity for graph convolutions to pass information through all molecules is O (b²). In contrast, each RNN in the All SMILES encoder can in principle pass information through the entire graph, so the computational complexity remains (M b). This analysis however does not take into account the practical speed up of sparse parallelizable graph convolutional operations in comparison with the sequential nature of RNNs.
3.2 LATENT SPACE OPTIMIZATION
Unlike many models that apply a sparse Gaussian process to fixed latent representations to predict molecular properties (Dai et al., 2018; Jin et al., 2018; Kusner al., 2017; Samanta al., 2018), the All SMILES VAE jointly trains property regressors with the generative model (as do Liu et al., 2018). We use línear regressors for the log octanol-water partition coefficient (logP) and molecular weight (MW), which have unbounded values; and logistic regressors for the quantitative estimate of drug-likeness likeness (QED) (Bickerton al., 2012) and twelve binary measures of toxicity (Huang et al., 2016: Mayr et al., 2016), which take values in [0, 1]. We then perform gradient-based optimization of the property of interest with respect to the latent space, and decode the result to produce an optimized molecule.
(GémezzBombarce et al. (2018) jointly train regressor, hut still optimize using a Gaussian process.

---

Under review as a conference paper at ICLR 2020
Naively, we might either optimize the predicted property without constraints the latent space, or find the maximum a posterioni (MAP) latent point for a conditional likelihood over the property that assigns greater probability to more desirable values. However, the property regressors and decoder are only accurate within the domain in which they have been trained: the region assigned high probability mass by the prior. For n-dimensional standard Gaussian prior, almost all probability mass lies in a practical support comprising a thin spherical shell of radius vn- 2017, Gaussian Annulus Theorem). With linear or logistic regressors, predicted property values increase monotonically in the direction of the weight vector, so unconstrained property maximization diverges from the origin of the latent space. Conversely, MAP optimization with a Gaussian prior is pulled towards the origin, where the density of the prior is greatest. Both unconstrained and MAP optimization thus deviate from the practical support in each layer of the hierarchical prior, resulting in large prediction errors and poor optimization.
- I (Blum et al.,
We can use the reparametrization trick (Kingma & Welling, 2013; Rezende al., 2014) to map our autoregressive prior back to a standard Gaussian. The image of the thin spherical shell through this reparametrization still contains almost all of the probability mass. We therefore constrain optimization to the reparametrized n 1 dimensional sphere of radius vn- 1 for each n n-dimensional layer of the hierarchical prior by optimizing the angle directly. Although the reparametrization from the standard Gaussian prior to our autoregressive prior is not volume preserving, this hierarchical radius constraint holds us to the center of the image of the thin spherical shell. The distance to which the image of the thin spherical shell extends away from the highly nonlinear function of the previous layers. We describe this hierarchical radius constraint in more detail, and provide pseudocode, in Appendix B.3.
dimensional sphere at its center is a
RESULTS
We evaluate the All SMILES VAE standard 250,000 and 310,000 element subsets (Gómez- Bombarelli et al., 2018; Kang & Cho, 2018) of the ZINC database of small organic molecules (Irwin al., 2012: Sterling & Irwin, 2015). We also evaluate on the Tox21 dataset (Huang et al., 2016; Mayr et al., 2016) in the DeepChem package (Wu et al., 2018), comprising binarized binding affinities of 7831 compounds against 12 proteins. For further details, see Appendix A. Additional experiments, including ablations of novel model components, are described in Appendix C.
The full power of continuous, gradient- based optimization can be brought to bear on molecular properties given bijection between molecules and contractible regions of a latent space, along with a regressor from the latent space the property of interest that is differentiable almost every everywhere. Such bijection is challenging to confirm, since it is difficult to find the full latent space preimage of a molecule implicitly defined by a mapping from latent space to SMILES strings, such as our beam search decoder. As necessary condition, of molecules to latent space and back again, and that random samples from the prior distribution in the latent space map valid molecules. The former is required for injectivity, and the latter for surjectivity, of the mapping from molecules to contractible regions of the latent space.
confirm that it is possible to map from the space
Using the approximating posterior the encoder, but always selecting the mean of each conditional Gaussian distribution, and using a beam search over the conditional likelihood as the decoder, 87.4% + 1% of a held-out test set of ZINC250K (80/10/10 train/val/test split) is reconstructed accurately. With the same beam search decoder, 98.5% + 0.1%% of samples from the prior decode to valid SMILES strings. We expect that enforcing grammatical constraints in the decoder LSTM, as described in Appendix D, would further increase these rates. All molecules decoded from set of 50,000 independent samples from the prior were unique, 99. 99.958% were novel relative to the training dataset, and their average synthetic accessibility score (Ertl & Schuffenhauer, 2009) was 2.97 + 0.01, compared to 3.05 in the ZINC250k dataset used for training.
4.1 PROPERTY PREDICTION
Ultimately, would like optimize molecules for complicated physical properties, such as fluorescence quantum yield, binding affinity to selected receptors, and low toxicity. Networks can only be trained to predict such physical properties if their true values are known an appropriate
Thii generalizes the slerp interpolations of Bombarell et al. (2018) to optimization.
7

---

Under review as a conference paper at ICLR 2020
training dataset. While simple properties can be accurately computed from first principles, properties like drug efficacy arise from highly nonlinear, poorly characterized processes, and can only be accurately determined through tumenconuuiing and expensive experimental measurements. Since such experiments can only be performed on a small number of molecules, we evaluate the ability of the All SMILES VAE perform semi-supervised property prediction.
As Figure 5 and Table 5 in Appendix demonstrate, we significantly improve the state-of-the-ard in the semi-supervised prediction of simple molecular properties, including the log octanol-water partition coefficient (logP), molecular weight (MW), and quantitative estimate of drug-likeness (QED) (Bick- erton al., 2012), against which many algorithms have been benchmarked. We achieve similar improvement in fully supervised property prediction, shown in Table I, where we compare extended connectivity fingerprints (ECFP; Rogers Hahn, 2010), the character VAE (CVAF; Goonez-oombarele et al., 2018), and graph convolutions (Duvenaud et al., 2015). Table in Ap- pendix documents an even larger improvement compared models that use sparse Gaussian process for property prediction. We also surpass the state-of-the-arr in toxicity prediction the Tox21 dataset (Huang et al., 2016; Mayr et al., 2016), as shown in Table despite refraining from ensembling model, or engineering features using expert chemistry knowledge, in previous state-of-ther thtt--rrtthe-ttt methods (Zaslavskiy et al., 2019).
Accurate property prediction only facilitates effective optimization if the true property value is smooth with respect to the latent space. In Figure 6a, we plot the true (not predicted) logP over a densely sampled 2D slice of the latent space, where the axis is aligned with the logP linear regressor.
GraphCorn
0.1
MAE
0.02
All SMILES
0.01
0.25
0.05
0.2
0.5
0.05
0.1
0.05
0.1
Fraction labeled
Fraction labeled
Fraction labeled
(a) LogP
(b) MW
(c) QED
Figure 5: Semi-supervised mean absolute error (MAE) + the standard deviation across ten replicates for the log octanol-water partition coefficient (a), molecular weight (b), and the quantitative estimate drug-likeness (c; Bickerton et al., 2012) on the ZINC310K dataset. Plots are log-log; the All SMILES MAE is a fraction of that of the SSVAE (Kang & Cho, 2018) and graph convolutions (Kearnes et al., 2016). Semi-supervised VAE (SSVAE) and graph convolution results are those reported by Kang & Cho (2018).
72
35
2.5
0.
Preticnd pendlidd logf
Ostiniaaio Soep (-50)
(a) True logP over 2D slice latent space
(b) Predicted and true logP over optimization
Figure 6: Dense decodings of true logP along local 2D sheet in latent space, with the axis aligned with the regressor (a), and predicted and true penalized logP across steps of optimization (b).
8

---

Under review as a conference paper at ICLR 2020
Table 1: Fully supervised regression on ZINC250K (a), evaluated using the mean absolute error; and Tox21 (b), evaluated with the area under the receiver operating characteristic curve (AUC- ROC), averaged over all 12 toxicity types. Aside from All SMILES, results are those reported by ECFP: (Rogers& & Hahn, 2010), CVAE: (Gómez-Bombarell et al., 2018), GraphConv: (Duvenaud et al., 2015), Graph Conv Super Node (SN): (Li et al., 2017), PotentialNet: (Feinberg et al., 2018), and ToxicBlend: (Zaslavskiy et al., 2019). The ablation of atom harmonization is also evaluated on the Tox21 dataset.
(a) ZINC250K
(b) Tox21
MODEL
MAE LOGP
MAE QED
MODEL
AUC-ROC
ECFP
0.38
0.045
GRAPHCONV + SN
0.854
CVAE
0.15
0.054
POTENTIALNET
0.857 + 0.006
CVAE ENC
0.13
0.037
TOXICBLEND
0.862
GRAPHCONV
0.05
0.017
All SMILES (no harmonization)
0.864
0.003
All SMILES
0.005 + 0.0006
0.0052
:: 0.0001
All SMILES
0.8751 :: 0.0008
4.2 MOLECULAR OPTIMIZATION
We maximize the output of our linear and logistic property regressors, plus a log-prior regularizer, with respect to the latent space, subject to the hierarchical radius constraint described in Section 3.2. After optimizing in the latent space with ADAM, we project back to a SMILES representation of a molecule with the decoder. Following prior work, we optimize QED and logP penalized by the synthetic accessibility score and the number of large rings (Dai et al., 2018: Jin al., 2018; Kusner et al., 2017; Samanta et al., 2018; You et al., 2018; Zhou et al., 2018). Figure 6b depicts the predicted and true logP over optimization trajectory, while Table 2 compares the top three values found amongst 100 such trajectories to the previous state-of-the-an 4 The molecules realizing these property values are shown in Figure 7. Leaving the KL term in the ELBO unscaled by the number of SMILES strings in the decoder reduces the regularization of the latent space embeddings, allowing latent space optimization search a wider space of molecules that are less similar to the training set, as shown in Figure 14 of Appendix C.3.
Table 2: Properties of the top three optimized molecules trained ZINC250k. Other results
taken from JT- VAE: (Jin et al., 2018), GCPN: (You et al., 2018), MolDQN: (Zhou et al., 2018), and CGVAE: (Liu al., 2018). Following prior work, penalized logP is normalized by the statistics of the Zinc250k dataset.
MODEL
PENALIZED LOGP
MODEL
QED
JT-VAE
5.30, 4.93, 4,49
JT-VAE
0.925, 0.911, 0.910
GCPN MOLDQN
7.98, 7.85, 7.80
CGVAE
0.938, 0.931, 0.880
8.93, 8.93, 8.91
GCPN
0.948, 0.947, 0.946
ALL SMILES
12.31, 12.13, 12.01
MoIDQN
0.948, 0.948, 0.948
All SMILES unsealed)
29.80, 29.76, 29.11
All SMILES
0.948, 0.948, 0.948
4.3 ABLATION OF MODEL. COMPONENTS
In Table 3, we progressively ablate model components to demonstrate that all elements of the All SMILES architecture contribute to building a powerful fixed-length representation of molecules, rather than their particular SMILES string instantiations. We evaluate the effect these ablations on the mean absolute error (MAE) of logP and QED predictions, well the percentage of samples
"hoou et al. (2018) appear to report unnormalized penalized logP values: 11.84, 11.84, 11.82. In Table 2, we recompute normalized values for their best molecules. Recently, Winter et al. (2019b) reported molecules with penalized logP large as 26.1, but train an enormous, non-standard datasct of 72 million compounds aggregated from the 7INC15 and PubChem databases.

---

Under review as a conference paper at ICLR 2020
(a) Molecules with top three penalized logP values
(b) Molecules with the top three QED values
Figure 7: Molecules produced by gradient-based optimization in the All SMILES VAE.
from the prior that decode valid SMILES strings (Val) and the percentage of molecules that are reconstructed accurately (Rec acc). In all cases, we use the the mean of each conditional Gaussian distribution, and a beam-search decoder.
No ATOM HARMONIZATION removes the pooling amongst each instance of an atom across SMILES strings in the encoder, depicted in Figure As a result, the multiple SMILES inputs are processed independently until the final max pooling over GRU hidden states. A random SMILES string is chosen to serve as input to the attention mechanisms of the approximating posterior. Table 1b shows the significant effect of this ablation on toxicity prediction, demonstrating the importance of atom harmonization for nonlinear properties of the entire molccule, in contrast to the quasi-lineat logP and QED reported in Table 3. We extend this process in ONE SMILES ENC by only feeding a single SMILES string to the encoder, although the decoder still reconstructs multiple disjoint SMILES strings. ONE SMILES ENC/DEC (x) further reduces the size of the decoder set to one, but the encoded and decoded SMILES strings are distinct. Finally, ONE SMILES ENC/DEC (-) encodes and decodes a single, shared SMILES string. Except for ONE SMILES ENC/DEC (=), all of these ablations primarily disrupt the flow of messages between the flattened spanning trees, and induce similar, significant decay in performance. ONE SMILES ENC/DEC (=) further permits the latent representation to encode the details of the particular SMILES string. rather than forcing the representation of only the underlying molecule, and causes further reduction performance. This ablation uses canonical SMILES for encoding and decoding, as in Homez-Bombaree et al. (201 8).
We also observe meaningful contribution from the hierarchical approximating posterior. No POSTERIOR IEERARCHY, we move all latent variables to the first layer of the hierarchy, removing the succeeding layers. The prior a standard Gaussian, and there no attentional pooling over the atom representations. Further ablations of the hierarchical radius constraint reported in Appendix C.4.
Table 3: Effect of model ablation on fully supervised property prediction and generative modeling using the ZINC250k dataset.
ABLATION
MAE LOGP
MAE QED
VAL
REC ACC
FULL MODEL
0.005+0.0000
0.0052+0.0001
98.5+0.1
87.4+1.0
No ATOM HARMONIZATION
0.00840.004
0.007610.0005
97.6-0.2
84.0#0.4
ONE SMILES ENC
0.008+0.005
0.0073+0.0002
98.4+1..1
82.3+ 0.4
ONE SMILES ENC/DEC (+)
0.009+0.001
0.0091+0.0003
97.1+0.7
80.9+0.4
ONE SMILES ENC/DEC (=)
0.025+0.003
0.0115+0.0004
85.7+1
91.3+00.6
No POSTERIOR HIERARCHY
0.010+0.003
0.0051
98.2+00.5
85.2+0.6
CONCLUSION
For each molecule, the All SMILES encoder uses stacked, pooled RNNs on multiple SMILES strings to efficiently pass information throughout the molecular graph. The decoder targets disjoint set of SMILES strings of the same molecule, forcing the latent space to develop a consistent representation for each molecule. Attentional mechanisms the approximating posterior summarize spatially diffuse features into fixed-length, non-factorial approximating posterior, and construct latent representation on which linear regressors achieve state-of-the-ae semi- and fully- supervsuee property prediction. Gradient-basse optimization of these regressor outputs with respect to the latent representation, constrained subspace near almost all probability in the prior, produces state-of-ther afrt optimized molecules when coupled with simple RNN decoder.
10

---

Under review as a conference paper at ICLR 2020
REFERENCES
Jeremy Appleyard, Tomas Kocisky, and Phil Blunsom. Optimizing performance of recurrent neural networks on gpus. arXiv preprint arXiv: 160400146, 2016.
Tristan Aumentado-Armstrrnng Latent molecular optimization for targeted therapeutic design. arXiv preprint arXiv:1809.0200 2018.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning align and translate. arXiv preprint FrXXiv:409.044 2014.
G Richard Bickerton, Gaia Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90, 2012.
Esben Bjerrum and Boris Sattarov. Improving chemical autoencoder latent space and molecular de generation diversity with heteroencoders. Biomolecules, 8(4):131, 2018.
Esben Jannik Bjerrum. Smiles enumeration as data augmentation for neural network modeling of molecules. arXiv preprini arXiv: 770300707, 2017.
Avrim Blum, John Hoperoft, and Ravi Kannan. Foundations of Data Science. June 2017. URL https: //ww http:://wwwwwccsooooo.o.omo foundations- www.daaaiaaaeeeeeeeeeae
Regine S Bohacek, Colin McMartin, and Wayne C Guida. The art and practice of structure-based drug design: molecular modeling perspective. Medicinal research reviews, 16(1):3-50, 1996.
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint urXiv:1406.1 1078, 2014.
J Clayden, N Greeves, S Warren, and P Wothers. Organic chemistry. Oxford University Press, 2001.
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational autoencoder for structured data. arXiv preprint urXiv:1802.087990 2018.
Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pp. 933-941. JMLR. org, 2017.
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs. arXiv preprint 1rxiv:1805.11991 2018
David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. Convolutional networks graphs for learning molecular fingerprints. In Advances in neural information processing systems, pp. 2224-2233 2015.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminfor- matics, 1(1):8, 2009.
Evan N Feinberg, Debnil Sur, Zhenqin Wu, Brooke E Husic, Huanghao Mai, Yang Li, Saisai Sun, Jianyi Yang, Bharath Ramsundar, and Vijay S Pande. Potentialnet for molecular property prediction. ACS central science, 4(11):1520-1530, 2018.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning-Volee Volume 70, pp. 1263-1272. JMLR. org, 2017.
Rafael
Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato. Benjamín Sánchez- Lengelinn Dennis Sheberla, Jorge Aguilera- paalterra-prrrauureee, Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 4(2):268-22 222.266-776. 2018.
11

---

Under review as a conference paper at ICLR 2020
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik, Objective-reinforee generative adversarial networks (organ) for sequence generation models. arXiv preprint arXiv:1705. 2017.
Anvita Gupta, Alex T Müller, Berend JH Huisman, Jens A Fuchs, Petra Schneider, and Gisbert Schneider. Generative recurrent networks for de novo drug design. Molecular informatics, 37(1-2): 1700111, 2018.
David K Hammond, Pierre Vandergheynst, and Rémi Gribonval. Wavelets on graphs via spectral graph theory. Applied and Computational Harmonic Analysis, 30(2):129-11 2011.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Sepp Hochreiter and Jürgen Schmidhuber, Long short-term memory. Neural computation, 9()):
1735- 1780, 1997.
Ruili Huang, Menghang Xia, Dac-Trung Nguyen, Tongan Zhao, Srilatha Sakamuru, Jinghua Zhao, Sampada Shahane, Anna Rossoshek, and Anton Simeonov. fox2lchallenge build predictive models of nuclear receptor and stress response pathways as mediated by exposure to environmental chemicals and drugs. Frontiers Environmental Science, 3:85, 2016.
Yirang Im. Mounggon Kim, Yong Joo Cho, Jeong-A Seo, Kyoung Soo Yook, and Jun Ycob Lee. Molecular design strategy of organic thermally activated delayed fluorescence emitters. Chemistry of Materials, (115):19461999 2017.
Sergey loffe. Batch renormalization: Towards reducing minibatch dependence in batch-normalized models. In Advances in neural information processing systems, pp. 1945-1953, 2017.
John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan Coleman. Zinc: a free tool discover chemistry for biology. Journal of chemical information and modeling, 52(7): 1757-1768, 2012.
Natasha Jaques, Shixiang Gu, Dzmitry Bahdanau, José Miguel Hernández Lobato, Richard E Turner, and Douglas Eck. Sequence tutor: Conservative fine-tuning of sequence generation models with ki- control. In Proceedings of the 34th International Conference on Machine Learning, pp. 1645-1654. JMLR.org, 2017.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. arXiv preprint arXiv: / /802.04364, 2018.
Seokho Kang and Kyunghyun Cho. Conditional molecular design with deep generative models. arXiv preprint arXiv: 1000.0000.00,... 2018.
Steven Kearnes, Kevin McCloskey, Mare Berndl, Vijay Pande, and Patrick Riley. Molecular graph convolutions: moving beyond fingerprints. Journal of computer-aided molecular design, 30(8): 595-608, 2016.
Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Sigian He, Qingliang Li, Benjamin A Shoemaker, Paul Thiessen, Bo Yu, et al. Pubchem 2019 update: improved access chemical data. Nucleic acids research, 47(11):D102-D1109 2018.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. 1rXiv:1312.61 14, 2013.
arXiv preprint
Thomas Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: 02907, 2016a.
Thomas Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint 16Xiv:161 2016b.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. In Advances neural information processing systems, pp. 1097-1105, 2012.
12

---

Under review as a conference paper at ICLR 2020
Matt J Kusner, Brooks Paige, and José Miguel Hemándezz-obato. Grammar variational autoencoder. arXiv preprint arXiv:1703.01925, 2017.
Greg Landrum et al. Rdkit: Open-source cheminformatics, 2006.
Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Handwritten digit recognition with proppaaopaattion network. In Advances in neural information processing systems, pp. 396-404, 1990.
Junying Li, Deng Cai, and Xiaofei He. Learning graph-level representation for drug discovery. arXiv preprint arXiv: /woXiv::700..0 03744, 2017.
Yujia Li, Daniel Tarlow, Mare Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. arXiv preprini arXiv: 1.00499, 2015.
Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. Learning deep generative models of graphs. arXiv preprint arXiv:1803.03333 2018.
Jaechang Lim, Scongok Ryu, Jin Woo Kim, and Woo Youn Kim. Molecular generative model based on conditional variational autoencoder for de novo molecular design. Journal of cheminformatics, 10(1):31, 2018.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander Gaunt. Constrained graph variational autoencoders for molecule design. arXiv preprini arXiv: a 1805.09076, 2018.
Alessandro Lusci, Gianluca Pollastri, and Pierre Baldi. Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for drug-like molecules. Journal of chemical information and modeling, 53(7):1563-1575, 2013.
Tengfei Ma. Jie Chen, and Cao Xiao. Constrained generation of semantically valid graphs via regularizing variational autoencoders. In Advances in Neural Information Processing Systems, pp. 7113-7122 2018.
Andreas Mayr, Günter Klambauer, Thomas Unterthiner, and Sepp Hochreiter. Deeptox: toxicity prediction using deep learning. Frontiers in Environmental Science, 3:80, 2016.
Jonas Mueller, David Gifford, and Tommi Jaakkola. Sequence to better sequence: continuous revision of combinatorial structures. In International Conference on Machine Learning, pp. 2536-2544, 2017.
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo
design through deep reinforcement learning. Journal of cheminformaties, 9(1):48, 2017.
Mariya Popova, Olexandr Isayev, and Alexander Tropsha. Deep reinforcement learning for de novo drug design. Science advances, 4(7))eaa 2018.
Evgeny Putin, Arip Asadulaev, Yan Ivanenkov, Vladimir Aladinskiy, Benjamin Sanchez-Lengelingg Alán Aspuru-Guzik, and Alex Zhavoronkov. Reinforced adversarial neural computer for de novo molecular design. Journal of chemical information and modeling. 58(6): 1204, 2018.
Edward O Pyzer-Knapp: Changwon Suh, Rafael Gómez-Bombarelli, Jorge Aguilera-Iparragun and Alán Aspuru-Guzik. What is high-throughput virtual screening? perspective from organic materials discovery. Annual Review of Materials Research, 45:195-216, 2015.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine Learning, pp. 1278-1286, 2014.
David Rogers and Mathew Hahn. Extendedd-comnectvity fingerprints. Journal of chemical information and modeling. 50(5):742-754, 2010.
Jason Tyler Rolfe. Discrete variational autoencoders. arXiv preprint arXiv:1609.02200 2016.
Seongok Ryu, Jaechang Lim, and Woo Youn Kim. Deeply learning molecular structure- structure-properss property relationships using graph attention neural network. arXiv preprint 1r0iv: 10988, 2018.
13

---

Under review as a conference paper at ICLR 2020
Bidisha Samanta, Abir De, Gourhari Jana, Pratim Kumar Chattaraj, Niloy Ganguly, and Manuel HonezzRodriguez NeVAE: a deep generative model for molecular graphs. arXiv preprint arXiv:1802.005283, 2018.
Benjamin Sanchez-Lengeling and Alán Aspuru-Guzik. Inverse molecular design using machine learning: Generative models for matter engineering. Science, 361(640):330-3655 2018.
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1):
120-131, 2017.
Martin Simonovsky and Nikos Komodakis. Graphvae: variational autoencoders. In International Conference on Artificial Neural Networks, pp. pp. 412-422. Springer, 2018.
Towards generation of small graphs using
Edward Snelson and Zoubin Ghahramani. Sparse gaussian processes using pseudo-inputs. In Advances in neural information processing systems, pp. 1257-1264, 2006.
Teague Sterling and John J Irwin. Zine 15-ligand discovery for everyone. Journal of chemical information and modeling, 55(11) (0:222223337, 2015.
Dagmar Stumpfe and Jurgen Bajorath. Exploring activity cliffs in medicinal chemistry: miniperspec- tive. Journal medicinal chemistry, (917)))222222442,, 2012.
Iyya Sutskever, Oriol Vinyals, and Quoc Le. Sequence sequence learning with neural networks. In Advances in neural information processing systems, pp. 3104-3112 31-2, 12, 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826, 2016.
Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. Order matters: Sequence to sequence for sets. arXiv preprint arXiv: 1.1/0999,, 2015.
David Weininger. SMILES, chemical language and information system. 1. Introduction to method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36 1988.
Robin Winter, Floriane Montanari, Frank Noé, and Djork-Aré Clevert. Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations. Chemical Science, 2019a.
Robin Winter, Floriane Montanari, Andreas Steffen, Hans Briem, Frank Noé, and Djork-Amé Clevert. Efficient multi-objective molecular optimization in a continuous latent space. Chemical Science, 2019b.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: A benchmark for molecular machine learning. Chemical science, 9(2):13-5330 2018.
Jiaxuan You, Bowen Liu, Rex Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. arXiv preprint arXiv:1806.0244 02473, 2018.
Mikhail Zaslavskiy, Simon Jégou, Eric W Tramel, and Gilles Wainrib. Toxicblend: Virtual screening of toxic compounds with ensemble predictors. Computational Toxicology, 10:81-88, 2019.
Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. arXiv preprint arXiv: /8/0 0.06888. 2018.
14

---

Under review as a conference paper at ICLR 2020
A DATASETS
SMILES strings, well as the true values of the log octanol-water partition coefficient (logP), molecular weight (MW), and the quantitative estimate trug-likeness (QED), are computed using RDKit (Landrum et al., 2006).
NH
(a) COCOCICNCCIONN CCINC(CNCIYOCOC
(b) CCCCC(CCCC)CCCC
Figure 8: Multiple SMILES strings of single molecule may be more dissimilar than SMILES strings of radically dissimilar molecules. The top SMILES string for molecule (a) is 30% similar to the bottom SMILES string by string edit distance, but 60% similar to the SMILES string for molecule (b).
A.1 ZINC
For molecular property optimization and fully supervised property prediction, we train the All SMILES VAE on the ZINC250k dataset of 250,000 organic molecules with between and 38 heavy atoms, and penalized logPs' from -13 to 5 (Góbez-Bommarelli et al., 2018). This dataset curated from subset of the ZINC12 dataset (Irwin et al., 2012), and available from https: / //githbb. com/ wwwaauspuuuuuge guzik-uz--- group/oooooooooo httpp/chheaaaaaa//aeeee vae. The distribution of molecular diameters in ZINC250K is shown in Figure
For semi-supervised property prediction logP, MW, and QED, train on the ZINC310K dataset of 310,000 organic molecules with between and 38 heavy atoms (Kang & Cho, 2018). This dataset is curated from the full ZINC15 dataset (Sterling & Irwin, 2015), and available from https: /uttpa//qiiee //tpt////tttt////// www/nyu-uee..o...... http://yyaalcoonttiionaallll mww.aaaaallleeeeeeeeeeeeeeeeeeee ssvae.
600000
500000
400000
Molecule Diameter
Figure 9: Histogram of molecular diameters in the ZINC250K dataset. The diameter is defined as the maximum eccentricity over all atoms in the molecular graph. The mean is 11.1; the maximum is 24. Typical implementations of graph convolution use only three to seven rounds of message passing (Duvenaud et al., 2015; Gilmer et al., 2017; Jin et al., 2018; Kearnes et al., 2016; Liu et al., 2018; Samanta al., 2018: You et al., 2018), and so cannot propagate information across most molecules this dataset.
'The log octanol-water partition coefficient minus the synthetic accessibility score and the number of rings with than six
15

---

Under review as a conference paper at ICLR 2020
A.2 Tox21
For the semi-supervised prediction of twelve forms of toxicity, we train on the Tox21 dataset (Huang al., 2016; Mayr et al., 2016), accessed through the DeepChem package (Wu et al., 2018), with the provided random train/validation/test set split. This dataset contains binarized binding affinities against up to proteins for 6264 training, 783 validation, and 784 test molecules. Tox21 contains molecules with up to 140 atoms, ranging from large peptides to lanthanide, actinide and other metals. Many these metal atoms are not present in any of the standard molecular generative modeling datasets, and there are metal atoms in the validation and test set that never appear in the training set. To address these difficulties, we curated an unsupervised dataset of 1.5 million molecules from the PubChem database (Kim al., 2018), which we will make available upon publication. To maintain commensurabiilty with prior work, this additional unsupervised dataset is only used on the Tox21 prediction task.
B EXTENDED MODEL ARCHITECTURE
The full All SMILES VAE architecture is summarized Figure 10. The evidence lower bound (ELBO) of the log- sigllikelioodd,
L E (()))) [logp(<>=)) DEL [[==)))]=<=))
the sum of the conditional log- liglliiessoods of x in Figure 10, minus the Kullback- divergence between the approximating posterior, (=][2), computed by node AP in Figure 10, and the prior depicted Figure 11.
Che((sanc))
X1
BiGRU
Lin
b
h!
LSTTM
X
[nco(CT)-1
elnoseiCC)
BiGRU
Lin
Enc
Enc²
AP
LSTM
alexCCledd
BiGRU
Lin
LSTM
Figure 10: Multiple SMILES strings representing single, common molecule are preprocessed by BiGRU and a linear transformation, followed by multiple encoder blocks as in Figures 2 and 3. The approximating posterior depicted in Figure then produces a sample from the latent state which is decoded into SMILES strings by LSTMs. Note that all SMILES strings, both the input and the output, are distinct. The encoder blocks also receive linear embedding of the original SMILES strings as input.
NN
NN
NN
11,0
Jr,
μ1,
Figure The prior distribution over [21, 22, z . is hierarchy autoregressive Gaussians, The conditional prior distribution of hierarchical layer given layers through --1, 1l,,((zzz,,, 22, Z2, 12....,,.111 Gaussian with mean y and log-variance log determined by neural network with input [21, 22, ;2-1]
In all experiments, approximating posterior uses four layers of hierarchy, with 128 hidden units in the mne-hidee--laye neural network that computes the attentional query vector. In practice, separate GRUs were used to produce the final hidden state for zj and the atom representations for z The single-layer LSTM decoder has 2048 hidden units. Training was performed using ADAM, with decaying learning
encoder stacks of depth three, with 512 hidden units in each GRU. The
16

---

Under review as a conference paper at ICLR 2020
rate and KL annealing. In all multiple SMILLE strings architectures, encoding and decoding which are selected with RDkit (Landrum et al., 2006).
5 SMILES strings for
In contrast to many previous molecular VAEs, do not scale down the term DKL [[2])))lp((2) in the ELBO by the number of latent units (Dai et al., 2018; Kusner et al., 2017). However, our loss function does include separate reconstructions for multiple SMILES strings of single molecule. For molecular optimization tasks, we usually scale up this KL term by the number of SMILES strings in the decoder, analogous multiple single-SMILES VAEs in parallel; we leave the KL term unscaled for property prediction.
B.1 GATED RECURRENT NEURAL NETWORKS
Convolutional neural networks on images (Krizhevsky et al., 2012; LeCun et al., 1990) leverage the inherent geometry of the visual domain to perform local message passing. At every spatial location of each layer, a convolutional network computes a message comprising the weighted sum of messages from the surrounding region in the preceding layer, followed by a point-wise nonlinearity. Each round of messages propagates information a distance equal to the convolutional kernel diameter multiplied by the current spatial resolution.
Recurrent neural networks, such as long short-term memories (LSTMs) (Hochreiter & Schmid- huber, 1997) and gated recurrent units (GRUs) (Cho al., 2014), model text, audio, and other one-dimensional sequences in an analogous manner. The kernel, comprising the weights on the previous hidden state and the current input, has a width of only two. However, the messages (i.e., the hidden states) are updated consecutively along the sequence, so information can propagate through the entire network single pass, substantially reducing the number of layers required. LSTTss and GRUs are ubiquitous in natural language processing tasks, and efficient GPU implementations have been developed (Appleyard et al., 2016).
Gated recurrent units (GRUs) are defined by (Cho et al., 2014):
[r, 2] = σ (z, [W,, W + h [U, U-] + [6,, b2)) h (1 2) h + tanh (x, + (r M-)
where r, 2, and h are row-vectors, [z, y] denotes the column-wise concatenation of and y, and the logistic function (zx) = (1+ and hyperbolic tangent are applied element-wise to vector argument x. The hidden state h, comprising the message from node is gated, weighted sum of the previous message h4-1 and the current input tt, both subject to an element-wise linear transformation and nonlinear (sigmoid) transformation. Specifically, the sum of the message from the input, x; WU-, and the gated message from the previous node, r O /4-1, is subject to linear transformation U and a pointwise nonlinearity. This is then gated and added to gated residual connection from the previous node.
Long short-teme teort-term memories (LSTMs) are defined similarly (Hochreiter & Schmidhuber, 1997):
[fe, te, 0:] (W, W., W.] + h U, Uo] [by, by, bol) Q = ft @ Ct-1 + i @ tanh (z.Wa 1-- + be)
h, 0, @ tanh (c.)
where f is the forget gate, i is the input gate, and is the output gate. LSTMs impose second hyperbolic tangent and gating unit on the nonlinear recurrent message, but nevertheless still follow the form of applying width-two kernels and pointwise nonlinearities to the input and hidden state.
In contrast, message passing in graphs is defined by (Duvenaud et al., 2015; Kearnes et al., 2016; Kipf& & Welling, 2016a; Li et al., 2015):
h (m)
(m) (EN((n)
Σ
W
where (n) is the set of neighbors of node n, for which there is an edge between and m (n)). and fxx is a pointwise nonlinearity such as a logistic function or rectified linear unit. This message passing, also called graph convolutions, be understood first-order approximation spectral
17

---

Under review as a conference paper at ICLR 2020
convolutions graphs (Hammond et al., 2011). (Kipf & Welling, 2016a) additionally normalize each message by the square root of the degree of each node before and after the sum over neighboring nodes. (Kearnes et al., 2016) maintain separate messages for nodes and edges, with the neighborhood of a node comprising the connected edges and the neighhorhood of an edge comprising the connected nodes. (Li et al., 2015) add gating analogous to GRU.
An LSTM, taking a SMILES string as input, can realize a subset of the messages passed by graph convolutions. For instance, input gates and forget gates can conspire ignore open-parentheses. which indicate the beginning of a branch of the depth-first spanning tree traversal. If they similarly ignore the digits that close broken rings, the messages along each branch of the flattened spanning tree are not affected by the extraneous SMILES syntax. Input and forget gates can then reset the LSTM's memory close-parees passe-paennteses, which indicate the end of a branch of the depth-first spanning tree traversal, and the return to previous node, ensuring that messages only propagate along connected paths in the molecular graph.
A set of LSTMs on multiple SMILES strings of single molecule, with messages exchanged between the LSTMs, can generate all of the messages produced by a graph convolution. Atom-based pooling between LSTMs on multiple SMILES strings of the same molecule combines the messages produced in each flattened spanning tree, allowing every LSTM to access all messages produced by a graph convolution. While LSTM decoder generating SMILES strings faces ambiguity regarding which set of SMILES strings representing a molecule to produce, this is analogous to the problem faced by graph-based decoders, discussed in Appendix D.2
B.2 BAHDANAU-STYE ATTENTION
The layers of the hierarchical approximating posterior after the first define conditional Gaussian distribution, the mean and log-variance of which are parametrized by an attentional mechanism the form proposed by (Bahdanau et al., 2014). The final encoder hidden vectors for each atom comprise the key vectors k, whereas the query vector q is computed by one-hidden-layer network of rectified linear units given the concatenation of the previous latent layers input. The final output of the attentional mechanism, c, is computed via:
tanh (gWa + k,U T
exp(e:)
a: Σ, exp(es)
Easks asks
The output of the attentional mechanism is subject to batch renormalization and linear transformation to compute the conditional mean and log- variancae of the layer.
B.3 LATENT SPACE OPTIMIZATION
As discussed the text section 3.2, optimization in the latent space performed in a region of space constrained to sphere, chosen so that the space around it contains almost all of the probability mass of the prior. This radius of the sphere is defined by the prior distribution, and is equal to /N -I for a standard Gaussian. However, due our nonlinear, autoregressive parametrization of the prior, this radius is not analytically accessible, and so we cannot apply it directly. We circumvent this difficulty by using the radius defined for each conditional gaussian the prior, ignoring interactions between each layer. The pseudocode for optimization in the latent space is shown in Algorithm 2. We project each layer of latent variables separately onto the radius defined by their conditional gaussian distribution, and then optimize with respect to the N 1 angles.
To further ensure that the optimization is constrained to well-trained regions of latent space, we add B log p(2) the objective function, where 3 is hyperparameter. Finally, to moderate the strietly monotonic nature of linear regressors, we apply an element-wiss hard tanh to all latent variables before the regressor, with linear region that encompasses all values observed in the training set.
To compare with previous work as fairly as possible, we optimize 1000 random samples from the prior to convergence, collecting the last point from each trajectory with valid SMILES decoding.
18

---

Under review as a conference paper at ICLR 2020
Algorithm 1: Initialize Angles
output Angular coordinates of latent variable sample the spherical shell with radius VN -1 // For each layer in the hicrarchy
for i + to K do
- (0,1)
- Tel V:
// project onto spherical shell
0. ToPolarCoords (6:)
end
return {0.))
Algorithm 2: Optimization in latent space with hierarchical radius constraint
input Property models: [f1. fm Prior distribution:
[p(2x|NNN (2)x))) p(2) (o)))))) p(z)),, Objective function: O(-)
output :Spherical coordinates of molecule in latent space with converged property values initialize {0:) <- Initialis iallalizzgngee ()
// The first layer oz the prior a standard gaussian
4o + 0, 00 1;
for f- to K do
// For each layer in the hierarchy
2; < ToCartesianCoordd (1:)
// Re-parametriiee standard gaussian variable conditional gaussian at position i in the hierarchy
0, 14
Mit 044 < NN, (#2i)
// Compute p and a the next level
end
// Optimize )0.) until the objective function 0() has converged (0: + -rradientDesceet (O(fs)) )
return {0T)]
From these 1000 points, we evaluate the true molecular property on the 100 points for which the predicted property value is largest. Of these 100 values, we report the three largest. However, optimization within our latent space is computationally inexpensive, and requires no additional property measurement data. We could somewhat improve molecular optimization at minimal expense by constructing additional optimization trajectories latent space, and evaluating the true molecular properties the best points from this larger set.
Molecular optimization is quite robust to hyperparameters. We considered ADAM learning rates in 0.01, 0.001, 0.0001) and 3 0.01, 0.001, 0.0001).
B.4 SUMMARY OF NOVEL CONTRIBUTIONS
Starting with the work of -oannezllBmbarlliii et al. (2018), previous molecular variational autoen- coders have used one consistent SMILES string as both the input to the RNN encoder and the target of the RNN decoder. Any single SMILES string explicitly represents only subset of the pathways in the molecular graph. Correspondingly, the recurrent neural networks in these encoders implicitly propagated information through only fraction of the possible pathways. Kipf & Welling (2016b), Liu et al. (2018), and Simonovsky & Komodakis (2018), amongst others, trained molecular VAEs with graph convolutional encoders, which pass information through all graph pathways in parallel, but at considerable computational expense. None of these works used enough layers of graph convolutions to transfer information across the diameter of the average molecule in standard drug design datasets. This is partially overcome by Lusci et al. (2013) who ensemble RNN-based representations of multiple directed-acyclic graphs of a single molecule for property prediction. The All SMILES VAE introduces the use of multiple SMILES strings of a single, common molecule as input to a RNN encoder, with pooling of homologous messages amongst the hidden representations associated with different SMILES strings. This allows information to flow through all pathways of the molecular graph, but can efficiently propagate information across the entire width of the molecule in single layer.
19

---

Under review as a conference paper at ICLR 2020
Bjerrum & Sattarov (2018) and Winter et al. (2019a) trained peuuuencett-sequeeee transcoders to map between different SMILES strings of the same molecule. These transcoders do not define an explicit generative model over molecules, and their latent spaces have no prior distributions. The All SMILES VAF extends this approach to variational autoencoders, and thereby learns a AMILES-derive generative model of molecules, rather than SMILES strings. The powerful, learned, hierarchical prior of the All SMILES VAE regularizes molecular optimization and property prediction. To ensure that molecular property optimization searches within the practical support of the prior, containing almost all of its probability mass, we introduce hierarchical radius constraint on optimization with respect to the latent space.
C EXTENDED RESULTS
We compare the performance of the All SMILES VAE to a variety of state-of-the-at art algorithms that have been evaluated standard molecular property prediction and optimization tasks. In partieu- lar, we compare to previously published results on the character/chemical VAE (CVAE) (Gómez- Bombarelli al., 2018) (with results reported in (Kusner et al., 2017)), grammar VAE (GVAE) (Kus- VAE (SD-VAE) (Dai et al., 2018), junction tree VAE (JT- VAE) (Jin et al., 2018), NeVAE (Samanta et al., 2018), semisupervised VAE (SSVAE) (Kang & Cho, 2018), graph convolutional policy network (GCPN) (You et al., 2018), molecule deep Q-network (MoIDQN) (Zhou et al., 2018), and the DeepChem (Wu et al., 2018) implementation of extended connectivity fingerprints (ECFP) (Rogers & Hahn, 2010) and graph convolutions (GraphConv) (Du- venaud et al., 2015; Kearnes et al., 2016; Wu et al., 2018). Extended connectivity fingerprints a fixed-length hash of local fragments of the molecule, used input conventional machine learning techniques such as random forests, support vector machines, and non-convolutional neural networks (Wu al., 2018). For toxicity prediction, also compare to PotentialNet (Feinberg et al., 2018), ToxicBlend (Zaslavskiy et al., 2019), and the results of (Li et al., 2017).
ner et al., 2017), syntax-directe
C.I RECONSTRUCTION ACCURACY AND VALIDITY
Previous molecular variational autoencoders have been evaluated using the percentage of molecules that are correctly reconstructed when sampling from both the approximating posterior g(2(x) and the conditional likelihood p(z|z) (reconstruction accuracy), and the percentage of samples from the prior p(=) and conditional likelihood p(<r|=) that valid SMILES strings (validity). While these measure have intuitive appeal, they reflect neither the explicit training objective (the ELBO), nor the requirements of molecular optimization. In particular, when optimizing molecules via the latent space, a deterministic decoder ensures that each point in latent space is associated with single set of well-defined molecular properties.
The All SMILES VAE is trained on a more difficult task than previous molecular VAEs, since the reconstruction targets are different SMILES encodings than those input to the approximating posterior. This ensures that the latent representation only captures the molecule, rather than its particular SMILES encoding, but it requires the decoder LSTM to produce a complex, highly multimodal distribution over SMILES strings. As result, samples from the decoder distribution are less likely to correspond to the input to the encoder, either due to syntactic or semantic errors.
To compensate for this unusually difficult decoding task, evaluate the All SMILES VAE using a beam search over the decoder distribution. That is, we decode to the single SMILES string estimated to be most probable under the conditional likelihood p(z|2). This has the added benefit of defining an unambiguous decoding for every point in the latent space, simplifying the interpretation of optimization in the latent space (Section 4.2). However, it renders the reconstruction and validity reported in Section 4 incommensurable with much prior work, which use stochastic encoders and decoders.
C.2 PROPERTY PREDICTION
Rather than jointly modeling the space of molecules and their properties, some earlier molecular variational autoencoders first trained an unsupervised VAE on molecules, extracted their latent
'The full decoder distrihution still used for training.
20

---

Under review as a conference paper at ICLR 2020
representations, and then trained sparse Gaussian process molecular properties a function of these fixed latent representations (Dai et al., 2018; Jin et al., 2018; Kusner et al., 2017; Samanta al., 2018). Sparse Gaussian processes parametric regressors, with the location and value of the inducing points trained based upon the entire supervised dataset (Snelson & Ghahramani, 2006). They have significantly more parameters, and corresponding more powerful, than linear regressors.
Molecular properties are only smooth function of the VAE latent space when the property regressor trained jointly with the generative model Bombaezll et al., 2018). Results using sparse Gaussian process on the latent space of an unsupervised VAE are very poor compared to less powerful regressors trained jointly with the VAE. Our property prediction two orders of magnitude more accurate than sparse Gaussian process regression on an unsupervised VAE latent representation, as shown in Table 4.
Table 4: Root-mean-spp -quare of the log octanol- partition coefficient (logP) on the ZINC250k dataset. Results other than the All SMILES VAE are those reported in the cited papers.
MODEL
RMSE
CHARACTER VAE (CVAE) (GÓMEZ-BOMBAREH)
1.504
ET AL., 2018; KUSNER ET AL., 2017)
GRAMMAR VAE (GVAE) (KUSNER ET AL., 2017)
1.404
SYNTAX-DIRECTED VAE (SD-VAE) (DAI ET AL.,
1.366
2018)
JUNCTION TREE VAE (JT-VAE) (JIN ET AL., 2018)
1.290
NEVAE (SAMANTA ET AL., 2018)
1.23
All SMILES
0.011 + 0.001
We report numerical results on semi-supervised property prediction, as well as comparisons from (Kang & Cho, 2018), in Table 5. Our mean absolute error is at least three times smaller than comparison algorithms on the log octanol-water partition coefficient (logP) and molecular weight (MW).
Table 5: Mean absolute error (MAF) of semi-supervised property prediction on the log octanol water partition coefficient (logP), molecular weight (MW), and the quantitative estimate of likgelikeness (QED) on ZINC310K dataset. Results other than the All SMILES VAE are those reported by (Kang & Cho, 2018).
MODEL
LABELED
MAE LOGP
MAE MW
MAE QED
ECFP
50%
0.180 + 0.003
9.012 + 0.184
0.023 + 0.000
GRAPHCONV
50%
0.086 0.012
4.506 0.279
0.018 = 0.001
SSVAE
50%
0.047 + 0.003
1.05 + 0.164
0.01 = 0.001
ALL SMILES
50%
0.007 0.002
0.21 = 0.07
0.0064 #0.0002
ECFP
20%
0.249 0.004
12.047 + 0.
0.033 0.001
GRAPHCONV
20%
0.112 + 0.015
4.597 + 0.419
0.021 = 0.002
SSVAE
20%
0.071 0.007
1.008 + 0.370
0.016 0.001
ALL SMILES
20%
0.009 + 0.002
0.33 +0.06
0.0079 ++00003
ECFP
10%
0.335 0.005
15.057 + 0.358
0.045 0.001
GRAPHCONV
10%
0.148 + 0.016
5.255 + 0.767
0.028 = 0.003
SSVAE
10%
0.090 0.004
1.444 0.618
0.021 0.001
ALL SMILES
10%
0.014 + 0.002
0.30
0.0126 + 0.0006
ECFP
5%
0.380 + 0.009
17.713 + 0.396
0.053 + 0.001
GRAPHCONV
5%
0.187 0.015
6.723 2.116
0.034 = 0.004
SSVAE
5%
0.120 0.006
1.639 0.577
0.028 + 0.001
ALL SMILES
5%
0.036 + 0.004
0.4 + 0.1
0.0217 + 0.0003
21

---

Under review as a conference paper at ICLR 2020
As visual demonstration of the accuracy of property prediction, in Figure 12 we show the predicted logP a 2D slice of latent space subject to the hierarchical radius constraint, alongside the true logP of the molecules decoded from this slice (identical to Figure 6a).
75
2.5
25
000
(a) True logP
(b) Predicted logP
PdP
Pd
bd
100
P0
Pa
bd
PQ
100
000
3d
9
9.
0000
6.6
P00
404
040
po0
000
4x
p00 oos
g0o
krg
Roct
5r0
poo
poo
wa
xa
10.0
0.0"
000
2 oo. 800
2 goo
2g
Ad
86
2
uoo wy goz doo
goor af oiior aadd 23o
oogo Goo goo og ~go ago ao à
dorp 2oro
2010
(c) Sheet of molecules
Figure 12:
Dense decodings of true logP (a) and predicted logP (b) along local 2D sheet in latent space, with the axis aligned with the trained logP regressor. We also display coarse sampling of the molecules corresponding the logP heatmap (c).
Pathways on which activity (active or inactive) is assessed for the Tox21 dataset include seven nuclear receptor signaling pathways: androgen receptor, full (NR-AR) androgen receptor, LBD (NR-AR- LBD); aryl hydrocarbon receptor (NR-AHR); aromatase AUR-AROMATASE); estrogen receptor alpha, LBD (NR-ER-LBD), estrogen receptor alpha, full (NR-ER); and peroxisome proliferator- activated receptor gamma PEE--PAAR-MMMMAA The Tox21 dataset also includes activity assess- ments for five stress response pathways: nuclear factor (erythroid-derivel 2i-like 2/antioxidant responsive element (SR-ARE); ATADS (SR-ATAT) ASRAAD)))); heat shock factor response element (SR-HSE); mitochondrial membrane potential (SR-MMP); and p53 (SR-p53). We report the area under the receiver operating characteristic curve (AUC-ROC) on each assay independently in Table 6. The average of these AUC-ROCs is reported in Table 1. We do not include the result of (Keames et al.,
22

---

Under review as a conference paper at ICLR 2020
2016) in Table 1, since it is not evaluated the same train/valiidation/eest split of the Tox21 dataset, and so not commensurable.
Table 6: Area under the receiver operating characteristic curve (AUC-ROC) per assay on the Tox21 dataset.
NR-AR
NR-AR-LBD
NR-AHR
NR-AROMATASE
NR-ER
NR-ER-LBD
0.864
0.921
0.909
0.908
0.719
0.811
SE-PPAR-GAMMA
SR-ARE
SR-ATADS
SR RRE
SR-MMP
SR-p53
0.935
0.860
0.870
0.901
0.927
0.882
C.3 MOLECULAR OPTIMIZATION
We present an optimization trajectory for the quantitative estimate of stug-likeness, (QED) in Figure 13. For the molecules depicted in Figure 7, we scaled (g(2 [[[====][[[(())))) in the ELBO (Equation 1) of the All SMILES VAE by the number of SMILES strings in the decoder. This renders the loss function analogous to that of many parallel single-SMILES VAEs, but with message passing between encoders leading to shared latent representation. If we leave the KL term unscaled, latent space embeddings subject to less regularization forcing them to match the prior distribution. Optimization molecular properties with respect to the latent space therefore searches over a wider space of molecules, which are less similar to the training set. In Figure 14, we show that such an optimization for penalized log finds very long aliphatic chains, with penalized log P values as large 42.46.
0.85
0.90
0.85
Predioted QE1>
0.0
2.5
5.0
7.5
10.0
12.5
15.0
37.5
20.0
Optimization
Figure 13: Predicted (red line) and true (blue x's) quantitative estimate of drug-likeness (QED) over the optimization trajectory resulting in the molecule with the maximum observed true QED (0.948).
COONC
Figure 14: Molecules with the top three true penalized LogP values produced by gradient-based optimization subject to the hierarchical radius constraint in the All SMILES VAE, but with the KL term unscaled by the number of SMILES strings in the decoder. Molecules are shown as SMILES strings, wrapped across multiple lines, as they too large be properly rendered into image.
C.4 ABLATION OF HIERARCHICAL RADIUS CONSTRAINT
Table shows that the hierarchical radius constraint significantly improves molecular optimization. In contrast Table 2, optimization is performed penalized logP alone, without log prior regularizer.
23

---

Under review as a conference paper at ICLR 2020
This produces better results without the radius constraint, and constitutes conservative ablation experiment.
Table 7: Effect the hierarchical radius constraint on penalized logP optimization. Predicted penalized logP was evaluated 1000 optimization trajectories. From these, the true logP was evaluated on the 100 best trajectories, and the top three true penalized logPs are reported. Each optimization was repeated 5 times.
ABLATION
1ST BEST T.GPP
2ND BEST LOGP
3RD BEST L.GP
WITH RADIUS CONSTRAINT
17.0 + 3.0
16.0 + 2.0
14.8 0.3
WITHOUT RADIUS CONSTRAINT
8.5044 + 0.0
6.9526 + 0
5.36 = 0.05
D SMILES GRAMMAR CAN BE ENFORCED WITH A PUSHDOWN AUTOMATON
The subset of the SMILES grammar (Weininger, 1988) captured by (Dai et al., 2018) and (Kusner et al., 2017) is equivalent to the context-free grammar shown in Figure 15. This subset does not include the ability represent multiple disconnected molecules in single SMILES string, multiple fragments that are only connected by ringbonds, or wildcard atoms. element symbools includes symbols for every element in the periodic table, including the al aliphatic organic symbols.
chain - branched atto rest_of_chain
rest_of_chain -> bond? chain bond + '.'
"#' -
branched_atom atom ringbond* branch*
ringbond > bond digit? digit
branch + '(" bond? chain 'Y'
atom - aliphatic organic- - aromatic organic bracket atom
aliphaticoogaant 'B' C - 'N' '0' - P -p'-1 'F' CCI' - 'B'' - T
orggaicc > 'b' "c' 'n' '0'
_ ' '
bracket atom 'I' isotope? symbol chiral? hcount? charge? class? 'I' isotope - digit? digit? digit
symbol + element.symbolt
aromatic _symbols
aromatic. > 'c' - n'' "0" 'p' - 's' - 'se" 'as'
chiral @ @ @
"THHI' "TTH2" @@AL' "@AL2
@SPI' '@SP2' - "@SP3' @TBBI' .@TB2' ... *@ТB30'
@OHHI "OO2' "@OH30
hcount + 'H' digit?
charge 1 digit? '+' digit?
class - ':' digit? digit? digit?
digit '0' '1' 2' 3' '4
'5'
'6'
' 88 "9
Figure 15: Context-free grammar of SMILES strings
Productions generally begin with unique, defining symbol or set of symbols. Exceptions include bond and charge (both can begin with -), and wwommaic.ooggee (both include n, o, s, and p), but these pairs of productions never occur in the same context, and so cannot be confused. The particular production for chiral can only be resolved by parsing characters up the next production, but the end of chiral and the identity of the subsequent production can be inferred from its first symbol of the production after chiral. Alternatively, the strings of chi ral can be encoded monolithic tokens.
and aromatic ww..ttcbolllooooo is
24

---

Under review as a conference paper at ICLR 2020
Whenever there is choice between productions, the true production is uniquely identified by the next symbols. The only aspect of the SMILES grammar that requires more than a few bits of memory is the matching of parentheses, which can be performed in straightforward manner with a pushdown automaton. As a result, parse trees (Dai et al., 2018: 8; Kusner et al., 2017) need not be explicitly constructed by the decoder to enforce the syntactic restrictions of SMILES strings. Rather, the SMILES grammar can be enforced with a pushdown automaton running in parallel with the decoder RNN. The state of the pushdown automaton tracks progress within the representation of each atom, and the sequence of atoms and bonds. The set output symbols available to the decoder RNN is restricted those consistent with the current state of the pushdown automaton. ( and [ are pushed onto the stack when are emitted, and must be popped from the top of the stack in order to emit ) or respectively.
For example, in addition to simple aliphatic organic (B, C, N, O, S. CI, Br, or I) or aromatic organic (b, c, n, O, S, or p) symbols, an atom may be represented by a pair of brackets (requiring parentheses matching) containing sequence of isotope number, atom symbol, chiral symbol, hydrogen count, charge, and class. With the exception of the atom symbol, each element of the sequence is optional, but is easily parsed by finite state machine. i isotope, symbol, chi ral, heount, charge, and class can all be distinguished based upon their first character, so the position in the progression can be inferred trivially."
When parsing branched atom, all productions after the initial atom are r ringonnds until the first which indicates the beginning of branch. After observing a ) and popping the complementary ( off of the stack, the SMILES string is necessarily in the third component of a branched_at om, since only branched.at om can emit a branch, and only branch produces the symbol ). The next symbol must be a (, indicating the beginning of another branch, or one of the first symbols of gest.of.chatl since this must follow the branched.cco om in the chain production.
D.1 RINGBOND AND VALENCE SHELL SEMANTIC CONSTRAINTS
Similarly, the semantic restrictions of ringbond matching and valence shell constraints can be enforced during feedforward production of a SMILES string using pushdown stack and small (100-element) random access memory. Our approach depends upon the presence of matching bond labels at both sides of a ringbond, which is allowed but not required in standard SMILES syntax. We assume the trivial extention of the SMILES grammar to include this property.
ri ringbonds are constrained to come in pairs, with the same bond label both sides. Whenever a given ri ribonnd is observed, flip bit the random access memory corresponding to the ring number (the set of digits after the bond). When the ringbond bit flipped on, record the associated bend in the random access memory associated with the ring number, when the r ringbond bit is flipped off, require that the new bond matches the recorded bond, and clear the random access memory of the bond. The molecule is only allowed to terminate (rest_of_chee rather than bond? chain) when all ri ringbond bits are off (parity even). The decoder may receive as input which ringbonds are open, and the associated bond type, so it can preferentially close them.
produces e
The set of nested atomic contexts induced by chain n, branched_atom, and branch can be arbitrarily deep, corresponding to the depth of branching in the spanning tree realized by SMILES string. As a result, the set of SMILES symbols describing bonds to a single atom can be arbitrarily far away from t=he associated om. However, once a branch is entered, it must be traversed in its entirety before the SMILES string can return to the parent atom. For each atom, it is sufficient to push the valence shell information onto the stack as it is encountered. If the SMILES string enters a branch while processing an atom, simply push on a new context, with new associated root atom. Once the branch is completed, pop this context off the stack, and return to the original atom.
More specifically, each atom in the molecule completely described by a single branchedLatom and the bond preceding it (from the rest_of_chst that produced the branched.at Within each successive pair of bond and branchedLaton track the sum of the incom- ing wwwt.of.chainbeeeeoeeeeee.eeee
the internal r ringbond and branch bonds, and outgoing
symbol and hcount can both start with 'H', , but symbol is mandatory, so there is ambiguity.
25

---

Under review as a conference paper at ICLR 2020
rest.of_chain bond (from the succeeding rest.of.chaing on the stack. That is, each time new bond is observed from the atom, pop off the old valence shell count and push on the updated count. Require that the total be less than bound set by the atom; any remaining bonds are filled by implicity hydrogen atoms. Provide the numher of available bonds as input to the decoder RNN, and mask additional ri ringbonds and branches once the number of remaining available bonds reaches one (if there are still open ringbonds) or zero (if all r ringbonds are closed). Mask the outgoing bond, or require that rest_of wft..offciiiinn produce e, based upon the number of remaining available bonds.
D.2 REDUNDANCY IN GRAPH-BASED AND SMILES REPRESENTATION. OF MOLECULES
To avoid the degeneracy of SMILES strings, for which there are many encodings of each molecule, some authors have advocated the use of graph-based representations (Li et al., 2018; Liu et al., 2018; Ma et al., 2018; Simonovsky & Komodakis, 2018). While graph-based based processing may produce a unique representation in the encoder, it is not possible avoid degeneracy in the decoder. Parse trees (Dai et al., 2018; Kusner et al., 2017). junction trees (Jin et al., 2018), lists of nodes and edges (Li al., 2018; Liu al., 2018; Samanta et al., 2018), and vectors/matrices of node/edge labels (De Cao & Kipf, 2018; Ma et al., 2018; Simonovsky & Komodakis, 2018) all imply an ordering amongst the nodes and edges, with many orderings describing the same graph. Canonical orderings can be defined, but unless they are obvious to the decoder, they make generative modeling harder rather than easier, since the decoder must learn the canonical ordering rules. Graph matching procedures can ensure that probability within a generative model is assigned the correct molecule, regardless of the order produced by the decoder (Simonovsky & Komodakis, 2018). However, they do not eliminate the degeneracy in the decoder's output, and the generative loss function remains highly multimodal.
26


Reviews:
[{"review_id": "S1xGhOBM5H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors describie a novel variational autoencoder like method for molecules. Instead of using graph neural networks, the authors hava an approach based on SMILES which encode molecules as strings. To avoid the problem that any given molecule may be represented by multiple SMILES strings, the authors consider an encoder that makes use of several random SMILES representations of the input molecule. These are preprocessed using recurrent neural networks generating an average representation by pooling the representations generated with each SMILES sequence for each atom in the input molecule. This reduces the number of operations needed to share information across all the atom in the\nmolecule (from N^2 in graph neural networks to MN in the proposed approach with M different SMILES representations of the input molecule). The model decodes then into a disjoint set of SMILES strings different from those used at the\ninput. This enforces the model to learn a bijective mapping between molecules and latent representations. The model trains also jointly a property regressor, linear or logistic. They do constrained optimization in the latent space, satying\nwithin a reparameterized shell with most of the probability mass for the data. The proposed model can also do semi-supervised and supervised prediction tasks.\n\nClarity:\n\nThe paper is very clearly written and it is very easy to read. It contains a very detailed description of previous work.\n\nQuality:\n\nThe proposed model is very reasonable and well-motivated. The experiments performed are exhaustive and informative enough to show that the proposed model and algorithms are useful in practice.\n\nNovelty:\n\nThe proposed encoder/decoder model based on multiple SMILES representation is novel up to my knowledge.\n\nSignificance:\n\nThe experiments show that the proposed method can outperform previous ones. However, I miss additional evaluations using existing frameworks such as  Guacamol.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HylmnBCatS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a VAE-based method to predict molecular properties as well as to design a novel molecular architecture.\nThe method employs as a basic input the SMILES representation of molecules which are not well defined in terms of the representation, though.\nTo tackle the issue, the authors formulate the method based on (1) multiple inputs of SMILES strings, (2) the character-wise feature fusion across those multiple strings and (3) network training by multiple output targets of SMILES strings different from the input ones.\nAs a result, the method provides the fixed-length latent representation which is robust against the variations of the SMILES representation and is useful for predicting the molecular properties and optimizing the molecular design.\nThe experimental results on the tasks of the property prediction and molecular optimization using the benchmark datasets demonstrate the effectiveness of the proposed method in comparison with the others.\n\nThough this paper provides some contributions to the field of molecular graph representation, it contains flaws in presentation, lacking details of technical contents; thus, the paper is regarded as \"borderline\". I would like the authors to properly provide the details in the following points.\n\n* This paper lacks some important technical contents, making it hard to understand. \n- What is the actual form of the property predictor used in p(\\rho|z). And, in the first place of the paper, it would be better to explain what kind of and how many properties of the molecular are considered.\n- How is the decoder constructed and trained? Though the authors state that a single-layer LSTM is employed as the decoder, there is no clear description how to cope with the multiple outputs (decoder targets) in training the LSTM.\n- What does \\theta in p_\\theta(z) mean? In the VAE, p(z) works as a simple prior on z.\n- The description, especially in Sec. 3 for the proposed method, is rather poorly presented by using less amount of math. At least, the authors should first depict the overall architecture of the method through clarifying the above-mentioned technical points.\n- It is unclear how to apply the proposed method to the semi-supervised learning framework? Is it re-formulated in the SS-VAE [a] framework?\n- The comparison experiments seem to be inconsistent. The proposed method is compared with different methods in different datasets/tasks. Toward fair comparison, it should be evaluated in comparison with some baselines including such as JT-VAE consistently.\n\n[a] Kingma, D. P.; Mohamed, S.; Rezende, D. J.; Welling, M. Semi-Supervised Learning with Deep Generative Models. Proceedings of the 28th Annual Conference on Neural Information Processing Systems. 2014; pp 3581–3589.\n\n* The authors cope with the fluctuations regarding the SMILES representation by means of RNN. There, however, are some approaches to canonicalize the SMILES representation itself such as by canonical SMILES. The authors have to mention those other approaches and discuss the superiority of the method over them; hopefully the proposed method should be compared with the naive one simply combining VAE [Kang&Cho18] and canonical SMILES representation. And to further understand the difficulty in SMILES variations, it would be better to show the averaged cardinality of SMILES representation per molecular.\n\n\nMinor comment:\n* In aggregating the RNN features for each character in SMILES strings, it might be possible to incorporate some structural knowledge into the process; for example, \"C\"s in benzene share the identical feature representation through fusing all those features.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJeCVDLX5H", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors present a method All SMILES VAE that’s used for predicting chemical properties of small molecules and also for optimizing the structures of these molecules. The authors evaluate their model on the Zinc250K and Tox21 dataset and report that they are able to exceed the previous SOTA. This paper was well written, and did a good job with explanations and illustrations. \n\nThe central idea is to use a RNN to learn representations of strings encoding molecular structures (SMILE strings). SMILE strings are constructed by a DFS traversal over molecular graph structures. The authors choose to feed in distinct SMILE encodings of the same molecule in parallel, resulting in a stacked RNN architecture. They observe that since SMILE strings are DFS traversals of the molecular graph, propagation in the RNN corresponds to sequential message passing steps between nodes in the graph. This is in contrast to a graph neural network wherein all nodes simultaneously broadcast messages to their neighbors at every propagation step.\n\nWhile it’s interesting to be able to optimize molecules in the space of SMILE strings, the impact is less clear. The authors mention in Sec 3.1, that graph models e.g. GCNs have higher overall compute complexity O(b^2), compared to their method which has O(Mb); however this does not do complete justice. Most graph propagation operations benefit from heavily parallelizable sparse matrix operations on GPUs. Infact, they can actually be much faster than RNNs in practice since GCNs need only a fixed number of propagation steps (independent of the number of bonds).\n\nIn Sec 3.2, the authors describe their approach for constraining the space of molecular optimization. While this may lead to directed searches, it will prevent truly novel molecules from being synthesized. The authors will need to address and provide experiments with unconstrained search.\n\nIt was not clear if the authors implemented any of the baselines? It seems from the text of figure 5, that the SSVAE and GraphConv results have been taken directly from the paper. The authors can strengthen their claim by replicating the results of baselines and making sure that they agree.\n\nSome clarification questions:\n - For optimized/predicted strings, which are presumably novel molecules not in the dataset, how is the true chemical property  e.g. logP, determined?\n - In Figure 6, from steps 30 - 40, it seems that the predicted logP goes up but the true logP stays the same in a few cases. Why is this the case?\n\nOverall, I think that this paper has some interesting ideas and is well written. However, the novelty and impact of the model is somewhat lacking. If this were introducing a new application area to this field, then I think the case for acceptance could have been stronger, however there has already been a lot of work related to molecular property prediction/design. \n\nAlso, AC please note, I would have given the paper a score of 5, but Openreview only gave me a choice between 3 and 6. So, I went with 3, but please consider this to be a 5.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes All SMILES VAE which can capture the chemical properties of small molecules and also optimize the structures of these molecules. The model achieves significantly performance improvement over existing methods on the Zinc250K and Tox21 datasets. 

Overall it is a very solid paper - it addresses an important problem, provides detailed description of the proposed method and shows promising experiment results. The work could be a landmark piece, leading to major impacts in the field. However, given its potential,  the paper could benefit from major revisions of the draft. Below are some suggestions on improving the work:
1. The current version contains a lot of materials. It tries to strike the balance between machine learning methodology and details of the application domain. But the reality is that the lack of architecture details and some sloppy definitions of ML terms make it hard for readers to fully appreciate the methodology novelty. 

2. There is still room for improvement in experiments. As suggested in the review, more datasets should be used to evaluate the proposed model. Since it is hard to provide theoretic analysis of the proposed model,  extensive experiments should be provided. 

3. The complexity analysis is not fully convincing. Some fair comparison with the alternative approaches should be provided. 

In summary, it is a paper with big potentials. The current version is a step away from being ready for publication. We hope the reviews can help improve the paper for a strong publication in the future. 
Decision: Reject