LATENT OPTIMIZATION VARIATIONAL AUTOENCODER FOR CONDITIONAL MOLECULAR GENERATION
Authors: Kisoo Kwon
Jung-Hyun Park
Kuhwan Jeong
Sunjae Lee
Hoshik Lee
Venue:  (2021)
Keywords: latent optimization
Variational Autoencoder
molecular generation

Abstract: Variational autoencoder (VAE) is a generation algorithm, consisting of an encoder and a decoder, and the latent variable from the encoder is used as the input of the decoder. 
VAE is widely used for image, audio and text generation tasks. In general, the training of VAE is at risk of posterior collapsing especially for long sequential data. To alleviate this, modified evidence lower bounds (ELBOs) were proposed. However, these approaches heuristically control training loss using a hyper-parameter, and it is not way to solve the fundamental problem of vanilla VAE.
In this paper, we propose a method to insert an optimization step of the latent variable and alternately update the encoder and decoder of conditional VAE for maximizing ELBOs. 
In experiments, we applied the latent optimization VAE (LOVAE) on ZINC database, consisting of string representation of molecules, for the inverse molecular design. 
We showed that the proposed LOVAE achieves better performance than vanilla VAE in terms of ELBOs and molecular generation performance. In addition, the proposed method showed better performance in property satisfaction and property maximization tasks compared to existing works.

Under review as a conference paper at ICLR 2021
LATENT OPTIMIZATION VARIATIONAL AUTOENCODER FOR CONDITIONAL MOLECULE GENERATION
Anonymous authors Paper under double-bling review
ABSTRACT
Variational autoencoder (VAE) is a generation algorithm, consisting of encoder and a decoder, and the latent variable used as the input the decoder. VAE is widely used for image, audio and text generation tasks. In general, the training of VAE is at risk of posterior collapsing especially for long sequential data. To alleviate this, modified evidence lower bounds (ELBOs) were propsed. However, these approaches heuristically control training loss using a hyper-parameter, and are not way solve the fundamental problem of vanilla VAE. In this paper, we propose a method to insert an optimization step of the latent variable and alternately update the encoder and decoder for maximizing ELBOs. In experiments, we applied the latent optimization VAE (LOVAE) on ZINC dataset, consisting of string representation of molecules, for the inverse molecular design. We showed that the proposed LOVAE is more stable in the training and achieves better performance than vanilla VAE in terms ELBOs and molecular generation performance.
INTRODUCTION
Deep neural networks (DNNs) have demonstrated a dramatic performance improvement in various applications. Text extraction from image recognition, language translation, speech and natural language recogmition, and personal identification by fingerprint and iris have already achieved high accuracy (Wu et al. 2016 Devlin et al. 2018 Awad 2012. Nguyen et al. 2017). Recently, these applications became successful commercialized products.
For the purpose of generation of image, variational autoencoder (VAE) (Kingma & Welling, 2014). generative adversarial network (GAN) (Goodfellow et al. 2014), and reversible generative models (Dinh et al., 2015 2017; Kingma & Dhariwal (018)) were proposed and showed much progress (Bleicher et al. 22033 Phatak et al. 2009) Grathwhohl et al. 2018). These generative models were initially studied in image data and showed better performance than previous models. Since then, it has been extended in research area to generate new sentences (Iqbal & Qureshi] 2020) and to discover
new drugs (Chen al. 2018) and materials (Kim et al. 2018a).
Traditional materials research consists four steps: molecule design, physical or chemical property prediction, molecular synthesis, and experimental evaluation. These steps are repeated until the de- sired molecular properties of molecular structure are satisfied. Until now, trial-and-crror techniques based human knowledge have been widely used. However, they are time consuming and very expensive. In order to improve the traditional method, research for a high-throughhut computational screening (HTCS) (Bleicher et al., (200)) was conducted. However, this also had limitations such as high computational cost, predefined molecular structures by human knowledge, and low accuracy of simulation.
Unlike the traditional approach, inverse molecular design is an attempt to find novel molecules that satisfy desired properties from exploring a large chemical space (Sanchez-Lenge Lengelii- & Aspuru-Guzik 2018). It extracts knowledge of potential molecular structures and properties from accumulated molecular structure databases (PubChem, ZINC, etc.) and proposes new molecular structures that do not exist in their database (Bolton et al. 2008 Irwin et al. 2012). With the inverse molecular design, it is possible to save cost by conducting molecular synthesis and experimental evaluation only for molecular structures having desired properties instead of searching almost infinite chemical space.

---

Under review as a conference paper at ICLR 2021
With the development of machine learning techniques, the generative models such as GAN and VAE have been applied to inverse molecular design tasks in recent years (Sanchez- (Sanchez-Lengeling & Aspuru-Guzik] 2018 [hh et al. 2020; Yang al. 2020; Jin et al. 2018: Wang et 2020). In GAN, the discriminator tries to distinguish the molecular structure from the generator, and the generator tries generate similar molecular structure from the database, simultaneously. In VAF, encoder outputs latent variables using the molecular structure as an input, and decoder generates the original molecular structure using the latent variables from encoder.
However, there is a difficulty in training generative models, such as poor convergence. In GAN, it is difficult learn due to the alternative update method from the two player game situation (Goodfellow et al. 2014). Problems such as the two models oscillation without adversarial, or the fact that the parameters are no longer learned in a specific situation due to the mode collapsing phenomenon, are still problems to be solved. In VAF, evidence lower bound (ELBO) for training VAE models requires training both the reconstruction and the KL--divergence loss of the latent variables, but can cause a phenomenon called posterior collapsing. In order to prevent the posterior collapse, VAE (Higgins et al., 2017), Re-balancing-Van VAE Yan et al. 2019) and KL-annealing (Bowman et al. 2016) have been proposed.
In this paper, we propose a latent optimization VAB (LOVAE) that provides stable learning method by inserting the latent variable optimization technique in conditional VAE (cVAE) (Sohn et al. 2015). We apply two stages of the latent optimization to vanilla VAF. By first training encoder once, the latent variable has been optimized in the direction of reducing the training loss in the same input data. After a reparameterization of the latent variables from the updated encoder, an additional latent optimization was applied. Our proposed method, LOVAE, was compared and verified in the inverse molecular design task, and a drug-like molecular structure (ZINC dataset (Sterling & Irwin 2015)) was used as database. We show the proposed method outperform vanilla VAE in terms of reconstruction loss and ELBO, which are training indicators. In addition, it showed more improved appearance generation phase. Consequently, generating molecules of LOVAA showed higher uniqueness, novelty ratio, and target property satisfaction than some of previous approaches. And, also, LOVAE generated molecules that showed higher value of penalized LogP property than the existing methods.
2 RELATED WORK
Inverse molecular design The inverse molecular design based on human-knowledge is very time consuming and relies on the intuition of the researcher, so many researchers recently tried solve it through simulation method and others. HTCS (Bleicher et al. 2003). One of major simulation method, is an automated computational process that can rapidly identify active compounds, antibodies or genes, and the results provide starting points for many chemical works such drug and material design (Chen et al. 2018: Kim et al. 2018a). HTCS uses a kind of brute-force method for searching and analyzing the desired chemical characteristics of molecule using combination of hundreds or tens of millions of active compounds, but this can be disadvantage because the absolutely large amount of resources used to find the desired goals (Phatak al., (009). Recently, to solve the problems, artificial intelligence approaches have been widely applied in the field of the molecular design under the name of the inverse molccular design. The VAE and GAN are typical generative models and they have been applying to the inverse molecular design field (Sanchez- Lennell- & Aspuru- 2018 Jin et al. 2020 Yang et al. 2020 Simm et al. 2020).
For the inverse molecular design, various formats for simple representation of molecular structure information have been defined instead of atom's 's xyz coordinates. First, MDL format that represents 3D coordinate information and binding information between adjacent atoms together. Extended Connectivity Fingerprint (ECFP) (Rogers & Hahn 2010) and Simplified Molecular Input Line Entry System (SMILES) (Weininger et al.] 1989) are another representation for the molecular structure as sequential character string. And recently. the method of representing the molecular structure as a graph structure is also being researched. Among them, SMILES, string representation of molecules, is relatively easy to handle and has been showing a good performance.
Generative models The generative models such as GAN and VAE are very sensitive to the latent variables. In order words, the training of latent variables greatly affects the performance of the generative models. However, according way of dealing with latent variables, difficult problems
2

---

Under review as a conference paper at ICLR 2021
Vanilla VAE Update @i and f, simoltancously
LOVAE
Update
Additive Optimization
Update Decoder
loss
4rstal (x y:)
loss
Lyecon [x.co((X.y.2.
loss
Lewern (x.y, 2
dloss
Decodet
Decodar
Decodkr
Dacoder
Øloss de
ćloss
sampling
seupling
sampling
1,,
aloss Bip
Øloss + 3
Encoder
Eneoder
Liacoder
x
x
Figure 1: Comparison of the vanilla (conditional) VAE and LOVAE
have been occurred such posterior collapsing and mode collapsing in the training. As solution for this phenomenon, methods to adjust KL loss weight to a value other than have been proposed in VAE. In Kim al. (2018b), latent variable from encoder is optimized according to maximizing of ELBO, and both of the encoder and decoder parameters are updated as the optimized latent variable. In Zhang al. (2020), the reconstruction loss on the latent space is additive to the previous loss term. In GAN, there are several works applying the latent variable optimization. The method in Bojanowski al. (2017) optimizes the latent variable by assuming that the latent variable is learnable noise. In Wu et al. (2019), randomly sampled latent variables are performed a gradient descent in the direction of reducing the loss of GAN. After that, the parameters of the discriminator and generator are updated with the loss from the optimized latent variable 2. All of these approaches resulted in training stability and performance improvement in GAN.
CONDITIONAL VAE
In the conditional VAE (Kang & Chol 2018 Kingma et al. 2014), the input variable x is assumed
to generated from a generative distribution (xly, z) conditioned on the output variable y and latent variable z. The prior distribution of z is assumed to be p(z) (Z (2 I). We use variational inference to approximate the posterior distribution z given x and y by
(g(z|x. y) = (N(z)p y). diag(a (x.
(1)
From the perspective of the auto-encoder, go (x x y) and po (xly, z) are called as an encoder and a decoder, respectively. Feed-forward neural networks used for M (x, y)) and (xx y).
The objective of the conditional VAE is to maximize ELBO, which is lower bound of the marginal log-likelihood:
log y) 2 Ega(zix.y) log (xly, z) - (- up to an additive constant, where KI. denotes the Kullback-Leihle divergence. Given random
sample z generated from the encoder qa y). define the total loss as
Liotal (x, y, = log Py x , + KL ( x) ll)))))
Then, -xt y,) is Monte Carlo approximation of ELBO equation Crecon (x, y,y,2, = px z), because can be regarded as reconstruction loss. In vanilla
2
We define
conditional VAE, parameters 0 and are jointly optimized to minimize Liotal-
string representation of a molecule called SMILES widely used to analyze molecular data (Higgins al.] 2017 Yan al.) 2019 Kang & Cho 20T8). To deal with string data like SMILES in conditional VAE, recurrent neural networks (Yan et al. 2019 Kang & Cho 2018) are used for the decoder Pa (x} z). Given target molecular property y, a new molecule having this property is
generated in the following way:
p(z), (x())
3

---

Under review as a conference paper at ICLR 2021
Algorithm I An update of the encoder and decoder in LOVAE
Update the Encoder
Generate z ~ q0 (z) (z|x, x. y). Calculate Leotal X, y, 2). Update d' + Φ using Lxotal(X,
# Additive Latent Optimization Generate z' Qer (0 xx,
Update z"
9 9 where @rreeo (x.y yzy
Update the Decoder
Calculate Erecon(X, y, yx ). Update I' < 0 using Crecon (x, y.
LATENT OPTIMIZATION VAE
At first, we have considered advancing VAE in terms of latent variable optimization. The problem we originally wanted to consider in this paper is that vanilla VAE training is not in an optimized process. The decoder is trained depending on the encoder output z. However, in the vanilla VAE, the latent value, which is the result of encoder before the update with the same input data is used for the decoder training. From the perspective of the decoder, when the same input data x is used, may be more effective calculate Ltotal by using the updated latent variable (z') passing through the updated encoder. Our proposed method, LOVAE, tried to solve this problem terms of the latent variable optimization. LOVAE uses the same input for learning the encoder and decoder, and (z') used for updating the decoder because the encoder is updated first. From this approach, Lotal becomes smaller than the vanilla VAE (Cootal y, z) > Leotal (x, y, z')). In addition, helped the decoder training by optimizing (z') in the direction of reducing the (Crotal(x, y, zz) one more in way that does not spoil the training of encoder and decoder. That is, better encoder and better latent variable can make the decoder even better.
To be more specific, the encoder updated as usual with the decoder while the decoder is fixed, and optimization of the latent variable from the encoder follows. Finally, the decoder is updated with the optimized latent variable. Updating encoder first has an effect similar to latent optimization. By this updating, more suitable can be created and this latent variable not only reduces the loss but also depends on the input data. brief comparison of the vanilla (conditional) VAE and LOVAE is described in Figure]] T
First, the encoder parameter á is updated to d' in the direction of reducing the total loss Liotat (x, y, z) where is generated using the current encoder parameter 6. Secondly, 2' is generated using the updated encoder parameter ø', and then is updated to z" in the direction of reducing the reconstruction loss frecou (x, y, 2') using the natural gradient descent method (Wu et al. 2019). Lastly, the decoder parameter is updated to l' in the direction of reducing the reconstruction loss frocon (x, y, zz" using the optimized z".
Since the optimized z" is used in the update of the decoder parameter, LOVAE is expected to achieve bigger ELBO and show stable convergence. This will be verified with numerical results in Section Note that latent optimization is only applied in the training, and the inference of LOVAE remains same with the vanilla one.
In summary, the whole process of an update is detailed in Algorithm I
EXPERIMENTAL RESULTS
5.1 EXPERIMENT SETUP
ZINC database Stterling & Irwin 2015) is database that organizes information about various compounds drug-like like molecules. ZINC contains 3D structural information of compound quality and molecular physical properties such molecular weight (molWt), partition coefficient (LogP),
4

---

Under review as a conference paper at ICLR 2021
Figure 2: Example of SMILES in ZINC dataset: COtlcchhZCCCC0000ccccccccc222220
0.016
0.030
0.014
0.014
0.025
0.012
fraction
0.010
fraction
fraction
0.008
0.008
0.015
0.006
0.006
0.010
0.004
0.002
0.000
0.000
0.000
250
350 molwt
450
-2
0.2
LogP
QED
Figure 3: Distribution of target properties
and quantitative estimation of likg-likeness (QED). In addition, this information is provided in conformity with various molecular expression formats such as SMILES, mol2, and 3D SDF used in the chemical field. SMILES molecules, in the form of ASCII string. It is possible to express complex graph-structured chemicals in the sequence form using simple rules. Figure 2 shows SMILES of molecule in ZINC DB, and we used two types of DB, 2NCC10K (Yan et al. al. 2017j. The vocabulary contains 39 different symbols [1, 2, 3, 4, 5, 6, 7, 8, 9, +, =, #, (, H. B, C, N. O, F, Si, P, S, CI, Br, Sn, I, c, o, p. \. /, @ @ @ ). The minimum, median, and maximum lengths of a SMILES string of ZINC310K are 8, 42, and 86, respectively. (In case of ZINC250K, 9, 44, and 120)
string representation to express chemical substances, such as
& Cho 2018) and ZINC250K (Kusner
The distribution of target properties in our sampled dataset, ZINC310K, shown Figure Median values of molwt, LogP, and QED are 359.02, 2.91, and 0.70, respectively.
Among the existing VAE variants, beta- VAE (Higgins et al. 2017), re-balancing VAE Yaan et al. 2019), and KL-annealing VAE (Bowman et al. 2016) control the weight of KL loss in the total loss equation 3 to achieve their own purpose, such as the disentanglement of latent variables, the avoidance of the posterior collapse, and the stability of the training. The proposed latent optimization technique is also applicable with those methods.
Unlike LOVAE, semi-amortized VAE (SA-VAE) in Kim al. (018b) updates latent variable without the encoder update phase and applied a momentum based optimization multiple times. After that, decoder and encoder parameters are updated according to the optimized latent variable. According to (Kim et al.) 2018b), SA-VAE utilized the encoder and latent optimization parts even in test phase. That is, SA-V VAE is not proper to our task, because needs the latent optimization part. On the other hand, LOVAE does not utilize the additive latent optimization in the inference phase.
Like LOVAE, there an existing study that learns the encoder first (He et al.) (2019). In the case of He et al. (2019), the encoder is updated several times until certain condition is satisfied, and different input data are used when updating encoder and decoder. LOVAE differs some ways. updates the encoder first, but updates it only once without any stop condition, and uses the same input data for the encoder and decoder, and performs the additive latent optimization with the reparameterization to help overall VAE learning. We think z' obtained by the updating encoder at the same input data is more natural and suitable for the decoder learning. In addition, we think that using the same input data for the training encoder and decoder is more effective in terms of the latent optimization than using different input data to the encoder and decoder. Also, Table shows that LOVAE requires less training time. We referred to the experimental results Kim et al. (018b). Although SA-VAE and
5

---

Under review as a conference paper at ICLR 2021
Table 1: Comparison of total training time, in terms of relative speed
vanilla VAE
LOVAE
He et al. (2019)
SA-VAE
Relative speed
1.00
1.75
2.20
9.91
Table Training loss of ZINC310K and ZINC250K
vanilla VAE
LOVAE
ZINC310K
recon, loss
KL loss
total loss
recon. loss
KL loss
total loss
original ELBO
20.86
4.38
25.23
17.30 7.72
6.51 15.35
23.80
reduced KL term
14.90
9.31
24.21
23.07
vanilla VAE
LOVAE
ZINC250K
recon. loss
KL loss
total loss
recon. loss
KI. loss
total loss
original ELBO
16.69 5.93
6.13 16.63
22.81 22.55
14.76 8.99
7.34 12.52
22.10
reduced KI. term
21.50
He et al. 2019) used Yahoo dataset from Yang et al. (017)) it was compared with the same method, vanilla VAE.
The basic model structure in this paper follows the general VAE structure for sequential data. The encoder has a bi-directional RNN structure, and the decoder is a umi-directional RNN structure Yan et al. 2219 Kang & Cho 2018). Each RNN structure was composed of three layers of GRU cells, and the dimension of the latent variable was set to 100. The hidden size of each GRU cells is 250, and the dimension of the properties is 3. A 103-dimensional vector in which a 100-dimensional latent variable and 3-dimensional property are concatenated inputted to the decoder. Vanilla VAE and LOVAE have the exactly same model structure, but only training strategy differs.
We used the Adam optimizer with 31 = 0.9, 32 0.999, and = 10-6, and a polynomial-based learning rate decay was applied. The initial and end learning rate for the learning rate decay were 0.001 and 0.0, respectively. In the case of max epoch for the training, several cases were tried. Among them, set a number of the max epoch that shows good performance. During the training, normalize each property value to have a mean O and standard deviation 1. For the additive latent optimization of LOVAE, a 0.001 and B = 5 were applied.
5.2 EVALUATION: VAE TRAINING PHASE
As defined Section 3] Lrotal of VAE is the sum of frewon and LKL. It can be thought that Ctotal is primarily important, and the importance of frecon and LKL can be determined depending on the purpose. For the evaluation, models initialized with different random seeds were trained in each algorithm, vanilla VAE and LOVAE. The vanilla VAE is based cVAE. The trained models were compared and analyzed with Liotal and Erecon of the train set. The loss of LOVAE was measured without the additive latent optimization. Table 2 shows the training results for vanilla VAE and LOVAE We tried each method five times and calculated the mean of the training losses. 'Original ELBO' utilized equation 3 as the total loss, and 'reduced KI. term' reduced the weight of LKL. of equation like Yan et al. 2019). In our experiments, when the weight of KI. loss was from 0.7 to 0.8, each method showed a smaller Liotal. It can be seen that LOVAE better than vanilla VAE in both Lrocon and Lrosal- In the results of 'original ELBO', Lreeon of LOVAE is on average 2.74 less than that of vanilla VAE. terms of LKL, LOVAE is 1.68 greater, but Liotal of LOVAE is 1.07 smaller. In case of 'reduced KL term', LOVAE also shows smaller Liotal- The relative improvement of tCtotal about 4.45% in the case 'original ELBO'.
5.3 EVALUATION: MOLECULAR GENERATION PHASE
The generated molecules from the generative model can be evaluated according to three criteria. The first one is validity. This means that the generated molecule has a sound structure. This can be

---

Under review as a conference paper at ICLR 2021
Table 3: Generative efficiency at each property (ZINC310K)
molwt
LogP
QED
average
ss VAE (Kng & Cho
2018
0.760
0.0.866
0,901
0.842
vanilla VAE
0.800
0.868
0.880
0.849
LOVAE
0.897
0.924
0.937
0.920
Table 4: Probability that the property value of the generated molecule falls within 10% and 5% of the error range of the condition value (ZINC310K)
vanilla VAE
LOVAE
molwt
LogP
QED
average
molwt
LogP
QED
average
within 10%
0.791
0.577
0.616
0.662
0.893
0.703
0.630
0.742
within 5%
0.789
0.377
0.413
0.526
0.879
0.450
0.414
0.581
determined using the RDKit package (Landrum). The second one is novelty. The purpose of the inverse molecular design is to find new molecules that have not discovered yet. A molecule is said to be novel if it is not in the train set. The third one is uniqueness. If the latent space is very narrow and latent variable z repeatedly sampled in a similar space, there much room for generating the same SMILES. That is, the more uniqueness, the better the generative model. The generated molecules are more meaningful if they satisfy all three criteria, validity, novelty, and uniqueness. In this paper, the ratio that satisfies all three criteria is defined model attempts to generate 1,000 molecules and has 600 molecules that satisfy validity, novelty, and uniqueness at once, the generative efficiency is 0.6.
generative efficiency. For example, if generative
For evaluation, three values were determined for each property as a condition for the generative model, and ZINC3 10K was used. It was determined be close to the median, lower 10%, and upper 10% value in our train set. In case of molwt, 360.0, 260.0, and 460.0 were used as the condition. For LogP and QED, (3.0, 1.5, 4.5) and {0.7, 0.5, 0.9] were chosen, respectively. The molecular generation was attempted 3,500 times in each condition value, and 31,500 molecules were generated total of 9 conditions. For the analysis, 'original ELBO' models were used. The results of the generative efficiency are shown in Table It can be seen that LOVAE has good generative efficiency and uniform performance in all properties.
5.4 EVALUATION: PROPERTY SATISFACTION
In addition to the generative efficiency, it is possible to use property satisfaction as a measure of performance for eVAE. is how many molecules with properties being close the target condition can be generated. For this, evaluation of property satisfaction was conducted based on two criteria. The first is the percentage that the property value of the generated molecule falls within 10% of the eиror range of the condition value. For example, if the target value of molwt is 360.0, it measures the percentage of the generated molecules whose property value lies between 324.0 and 396.0. The results of the experiment are shown in Table In all three properties, LOVAE showed higher property satisfaction. At 10% and 5% error property satisfaction, LOVAE showed a relatively 12. % and 10.5% improvement, respectively.
For the comparison with previous works using ZINC250K, we referred to You al. (2018b). that paper, property targeting task was performed, and specific ranges of molwt and LogP were considered. In our case, if the target range of molWt from 150 to 200, we conditioned LOVAE as 175. The target ranges are four like Table
Except for the target range -2.5 S LogP 220, LOVAE showed the best performance. Since percentages of the training data in the range -25 < LogP S -2.0 and S LogP S 5.5 are 0.28% and 1.30%, the first target range can be a bit more difficult. In this respect, the result LOVAE at the target range- -2.5 & LogP -2.0 seems make sense.
7

---

Under review as a conference paper at ICLR 2021
Table 5: Probability that the property value of the generated molecule falls within target range (ZINC250K)
LogP
molwt
target range
-2.5
-2.0
5.0 5.5
150
200
500 ~500
JT-VAL (Jin et 2018
0.113
0.076
0.007
0.160
ORGAN Guimaraes al. 2017
0.000
0,002
0.151
0,001
GCPN (You et al. 2018a
0.855
0.547
0.761
0.741
LOVAE
0.316
0.606
0.992
0.976
Table 6: Properties of the top three optimized molcules trained on ZINC250K
penalized LogP
QED
Тop
Top 2
Top 3
Top
Top
Top 3
Grammar VAE Kusner et al.
2017)
2.94
2.89
2.80
Revalancing VA E Yan et
2019)
5.32
5.28
5.23
GCPN You et al. 201 8a
7.98
7.85
7.80
0.948
0.947
0,946
JT-VAE E Jin
2018
5.30
4.93
4.49
0.925
0.911
0.910
AIISMILES Alperstein al.
2019)
16.42
16.32
16.21
0.948
0,948
0.948
molecularRNNN (Popova
2019)
10.34
10.19
10.14
0.948
0.948
0.947
graphAF (Shi al. 2020
12.23
11.29
11.5
0.948
0.948
0.948
LOVAE LOVAE with plogP
20.59
18.26
16.39
0.948
0.948
0.948
15.47
15.46
15.45
5.5 EVALUATION: PROPERTY MAXIMIZATION (PENALIZED LOGP AND QED)
In the many previous papers, property maximization task performed and it was evaluated on penalized LogP (pLogP) and QED Kusner al. 2017). QED a property with boundary range with [0. 1]. but the range of penalized logp is -0, oo). pLogP LogP penalized by the synthetic accessibility score (SA) and the number of large rings (cycle), pLogP logP SA cycle. can be thought extrapolation task because generative models have to create new molecule that is not in the range of the property values the training DB (The maximum pl.ogP training DB = 5.072). In order to find molecules which show the highest property,
of previous approaches utilized reward function or property regressors with sparse Gaussian process (Kusner et al. [0017] (Shi et al.) al. 2018). In our approach, LOVAE, we just conditioned by high value such as 30.0 and 0.98 for Log! and QED, respectively.
2020, Ktömez-Bombarde
Table shows the property maximization results. LOVAE generated the new molecules shown the highest penalized LogP and QED property. It is noteworthy that LOVAE showed good performance only with LogP condition without separate part like reward or property regressor. That is, was confirmed that LOVAF, which is conditional VAE type, works properly even in the extrapolation task. Top 3 molecules of each property are represented Fig. In addition, LOVAE with pLogP as condition was trained and verified. The performance worse than LOVAE with LogP, but since the number large rings can also be given as condition, the trend of the generated molecules was different (5)
toxoo
plogP 20.59
plogP 18.26
plogP 16.39
QED 0.948
QED 0.948
QED 0.948
Figure Samples of generated molecules LOVAE
8

---

Under review as a conference paper at ICLR 2021
plogP 15.47
plogP 15.46
plogP 15.45
Figure 5: Samples of generated molecules of LOVAE with condition of plogP
CONCLUSION
In this paper, LOVAE applying latent optimization technique to VAE algorithm is proposed. By training the encoder first, the latent variable has updated distribution in the direction of reducing the train loss. Additionally, z is updated before training the decoder in the direction of reducing the reconstruction loss. The training of decoder becomes This was applied and verified in the inverse molecular design task in ZINC dataset, and confirmed that it showed better appearance in the train loss, ELBO, and molecular generative performance than those of vanilla VAE.
efficient by utilizing the optimized
REFERENCES
Zaccary Alperstein, Artem Cherkasov, and Jason Tyler Rolfe. All smiles variational autoencoder. arXiv preprint 19t5.:9990.3333333 2019.
Ali Ismail Awad. Machine learning techniques for fingerprint identification: A short review. In International Conference on Advanced Machine Learning Technologies and Applications, pp. 524-531. Springer, 2012.
Konrad Bleicher, Hans-Joachim Böhm, Klaus Müller, and Alexander Alanine. Hit and lead generation: beyond high-throughput screening. Nature reviews Drug discovery, 2(5):369-378, 2003.
Piotr Bojanowski, Armand Joulin, David Lopez-Paz, and Arthur Szlam. Optimizing the latent space of generative networks. International Conference on Machine Learning, 2017.
Evan E Bolton, Yanli Wang, Paul A Thiessen, and Stephen H Bryant. Pubchem: integrated platform of small molecules and biological activities. In Annual reports in computational chemistry, volume pp. 217-241, Elsevier, 2008.
Samuel Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. Generating sentences from a continuous space. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pp. 10-21, 2016.
Hongming Chen, Ola Engkvist, Yinhai Wang, Marcus Olivecrona, and Thomas Blaschke. The rise of deep learning in drug discovery. Drug discovery today. 22(6):1241-12551 2018.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independet components estimation. In International Conference Machine Leurning (ICML), 2015.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. In International Conference Machine Learning (ICML), 2017.
Rafael Hoamez-Bommbrellt. Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato Benjamín Sánchez- Lengeling, Dennis Sheberla, Jorge Aguilera laailerra-prrrguurree Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzik, Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 4(2):268-217 2018.

---

Under review as a conference paper at ICLR 2021
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural informa- tion processing systems. pp. 1672-2680, 2014.
Will Grathwhohl, Ricky T. Q Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint arXiv:/810.01367, 2018.
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling, Carlos Outeiral, Padro Luis Cunha Farias, and Aian Espuru-Guzize Objective-reinfore generative adversarial networks (organ) for sequence generation models. arXiv preprint arXiv: 705. 2017.
Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Berg- King-Kirkpatrick. Lagging inference networks and posterior collapse in variational autoencoders. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with con- strained variational framework. In Advances in neural information processing systems, volume 2, pp. 6, 2017.
Touseef Iqbal and Shaima Qureshi. The survey: Text generation models in deep learning. Journal of King Saud University-Compe and Information Sciences, 2020.
John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan Coleman. Zinc: free tool to discover chemistry for biology. Journal of chemical information and modeling, 52(7): 1757-1768, 2012.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International Conference Machine Learning, 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. International Conference Machine Learning (ICML), 2020.
Seokho Kang and Kyunghyun Cho. Conditional molecular design with deep generative models. Journal of chemical information and modeling, (0(1):43-52 2018.
Kyungdoc Kim, Seokho Kang, Jiho Yoo, Youngchun Kwon, Youngmin Nam, Dongseon Lee, Inkoo Kim, Youn-Suk Choi, Yongsik Jung, Sangmo Kim, et al. Deep-learning-basce inverse design model for intelligent discovery of organic molecules. upj Computational Materials, 4(1):1-7, 2018a.
Yoon Kim, Sam Wiseman, Andrew C. Miller, David Sontag, and Alexander M. Rush. Semi- amortized variational autoencoders. International Conference on Machine Learning (ICML), 2018b.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In Proceedings of the International Conference on Learning Representations (ICLR), 2014.
Durk. P. Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible Ixl convolutions, In Advances in neural information processing systems, pp. 10215-10224, 2018.
Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi-supervised learning with deep generative models. In Advances in neural information processing systems, pp- 3581-3589, 2014.
Matt J Kusner, Brooks Paige, and José Miguel Hernández-Looato. Grammar variational autoencoder. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 1945- 1954. JMLR. org, 2017.
Greg Landrum. Rdkit: Open-source cheminformatics. URL http: wwwp//wwww.. www.ww.k.it.oo
Kien Nguyen, Clinton Fookes, Arun Ross, and Sridha Sridharan. Iris recognition with off-the-shelf cnn features: A deep learning perspective. IEEE Access, 8:18848-18888 2017.
10

---

Under review as a conference paper at ICLR 2021
Sharangdhar S Phatak, Clifford C Stephan, and Claudio N Cavasotto. High-throughput and in silico screenings in drug discovery. Expert opinion on drug discovery, 4(9): 2009.
Mariya Popova, Mykhailo Shvets, Junier Oliva, and Olexandr Isayev. Molecularrnn: Generating realistic molecular graphs with optimized properties. arXiv preprint S0II::1905.13372 2019.
David Rogers and Mathew Hahn. Extended-conneettivg fingerprints. Journal of chemical information and modeling, 50(5):742-75,, 2010.
Benjamin Sanchez-Lengeling and Alán Aspuru-Guzik. Inverse molecular design using machine learning: Generative models for matter engineering. Science, 36 361(6400):60-366 2018.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Gregor NC Simm, Robert Pinsler, and José Miguel Hermández-Lobato Reinforcement learning for molecular design guided by quantum mechanics. International Conference on Machine Learning (ICML), 2020.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. In Advances neural information processing systems, pp. 1483-3491, 2015.
Teague Sterling and John J Irwin. Zinc 15-ligand discovery for everyone. Journal of chemical information and modeling, (5(31))2324-2337 2015.
Hui-Po Wang, Wen- Hsi-H Peng, and Wei-Jan Ko. Learning priors for adversarial autoencoders.
APSIPA Transactions on Signal and Information Processing, 9, 2020.
David Weininger, Arthur Weininger, and Joseph Weininger. Smiles. 2. algorithm for generation of unique smiles notation. Journal of chemical information and computer sciences, 29(2):771011, 1989.
Yan Wu, Jeff Donahue, David Balduzzi, Karen Simonyan, and Timothy Lillicrap. Logan: Latent optimisation for generative adversarial networks. arXiv preprint arXiv:1912.00953 2019.
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's 's neural machine translation sys- tem: Bridging the gap between human and machine translation. arXiv preprint arXiv: (609.68144 144, 2016.
Chaochao Yan, Sheng Wang, Jinyu Yang. Tingyang Xu, and Junzhou Huang. Re-balancing variational autoencoder loss for molecule sequence generation. arXiv preprint arXiv: 1900.00698, 2019.
Kevin Yang, Wengong Jin, Kyle Swanson, Regina Barzilay, and Tommi Jaakkola. Improving molecular design by stochastic iterative target augmentation. International Conference on Machine Learning (ICML), 2020.
Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, and Taylor BergKirkpatrick. Improved variational autoencoders for text modeling using dilated convolutions. In In Proceedings of ICML, 2017.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in neural information processing systems, 2018a.
Jiaxuan You. Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in neural information processing systems, pp. 1411-1421, 201 8b.
Zijun Zhang, Ruixiang Zhang, Zongpeng Li, Yoshua Bengio, and Liam Paull. Perceptual generative autoencoders, International Conference on Machine Learning (ICML), 2020.
11


Reviews:
[{"review_id": "VBdK4XMpzo", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose extensions to VAE training to stabilize the learning process. First, the encoder is trained while the decoder is fixed. This is followed by an additive optimization of the latent variable from the encoder. Finally, the decoder is updated based on the updated latent variable from the encoder. They conduct experiments on molecule generation tasks to show the efficacy of their method. The paper is well written and easy to follow.\n\nMy main concern with this paper is novelty. Their method of pretraining the encoder is not particularly novel and has been applied in Natural Language Processing. For example, in He et al (ICLR 2019) [2], the encoder and decoder steps of the VAE are separated and more steps of the encoder are performed in each iteration. \nAlso in  another related work, Li et al (EMNLP 2019), they combine this method with another trick (KL-thresholding). The difference between the proposed method and these existing methods are not clear (and I believe are minor). In fact the method of He at al, can be considered as a generalization of the proposed method, with minor differences.\n\nIf most of the techniques proposed in the paper are well known in the NLP community, then the only novelty is the application of this technique to SMILES strings. It is not surprising at all that it is better than vanilla VAE training. Even in this case, I think the comparisons with existing work is not thorough. \n\nIs it possible to compare the generated molecules on a broader set of criteria, for example using the MOSES evaluation framework? [1]. This allows one to evaluate the efficacy of the method along different criteria like validity, novelty, uniqueness, FCD, etc and compare it with a broader set of methods. The code is available at https://github.com/molecularsets/moses\n\nI would request the authors to carefully position their work with related work mentioned above.\n\n[1] Polykovskiy et al. Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models\n\n[2] He et al. Lagging Inference networks and posterior collapse in Variational Autoencoders (ICLR 2019)\n\n[3] Li et al. A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text (EMNLP 2019)\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "SBnqctf5iUT", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a way to use conditional variational autoencoder setups to design materials conditioned on properties (labels). The problem is of some significance in the materials design space using generative models, using machinery from neural network modeling - RNNs, VAEs, GANs and optimization. \n\nThe writing is clear, and material is readable. However, I note the following issues with the content:\n\n1. Problem motivation not very clear\nI felt that the paper does not provide good enough context into the problem and had to piece it together from related work cited (e.g. [1], [2]). In particular, the work seems to be quite related to the work [1] by Kang and Cho, where they use a graphical model fashioned as a VAE, with the dependencies being the label or property y, the latent variable z and the molecule x - i.e. ELBO(log p(x, y)) in equation (2) of the paper. \n\nThe paper's claim is that they address problems with the vanilla conditional VAE setup. However, it seemed difficult to figure out what exactly the paper was trying to fix in the original VAE setup. They mention \"posterior collapse\", although there does not seem to be much evidence given (please clarify) to show these deficiencies in the setup - I assume that we should look at section 5.2, Table 1, but no mention of posterior collapse is made in these likelihood numbers. \n\nI would also like to quote the following lines from the beginning of section 4. It is unclear again what is meant by saying that the decoder is sensitive to the encoder results and the accuracy might vary ...\n\n\"Since the latent variable z generated from the encoder is inputted to the decoder to calculate Ltotal,\nthe encoder and decoder are closely related. If the encoder generates non-informative or highly\nvolatile z, then the decoder is difficult to reconstruct x from z. In this case, it may not be optimal to\nupdate the encoder and decoder simultaneously. The decoder is very sensitive to the encoder results,\nz, and the accuracy of the decoder training may vary according to the encoder results. In other words,\nthe result of the encoder can be optimized in advance in direction in which the decoder trains well.\nThat is, better encoder and better latent variable can make the decoder even better.\"\n\n2. The optimization process: Aside from the issues regarding the paper's motivation, the optimization procedure adopted does not seem to be very clear and could perhaps need clarification. The paper mentions a 'two step' optimization process, of first updating the encoder (keeping decoder fixed) and then updating the decoder. This is strongly suggestive of the wake-sleep algorithm by Hinton et al [2], but if that is so, the exposition would need considerable elaboration with equations, optimization objectives, and clarification on how the optimization machinery used solves the actual problems at hand (which I also had some difficulty following, as explained in the previous point). In other words, the connection between the technique adopted and the problem solved is not clear. \n\nI think the paper needs additional work in explaining the motivation of the problem with examples on why the original VAE setup is deficient and what the proposed fixes do to solve the problem at hand. \n\n[1] Kang and Cho: https://arxiv.org/pdf/1805.00108.pdf\n\n[2] Wake-sleep algorithm https://www.cs.toronto.edu/~hinton/csc2535/readings/ws.pdf\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "GuenAyI4Xb_", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a novel way to train variational auto-encoders and apply it to generate small molecules. The paper is well presented and the method easy to follow. Overall I think the goals of the article are somewhat disjoint, and it feels like the proposed method (LOVAE) should be applicable to standard datasets as well (e.g. CIFAR10), which would have helped the “story” of the paper. This is especially important since I find the motivation for the LOVAE reasonably weak, and there’s not much in terms of theoretical backing.\n\nSome further comments are as follows:\n* Glow isn’t really a good “general reference” on reversible generative models, many successful versions exist prior to it.\n* The QED metric seems to top out at 0.948, why does this happen? It doesn’t look very informative as is.\n* All images, notably Figure 5, should be in vector format. It’s currently hard to see what’s in there and it does not scale.\n* There are some language errors, e.g. “the training of decoder”. It might be worth having another look at language throughout.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "4OC-1-m22hi", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThe paper proposes a latent optimization VAE (LOVAE) to insert an optimization step of the latent variable and alternately update the encoder and decoder of conditional VAE. The additional latent optimization step optimizes phi with respect to the reconstruction loss. It makes sense the reconstruction loss is low in table 1. \n\n\n1: Although the adaptation of latent optimization to cVAE is interesting and can improve the performance, the idea or the contribution is not significant enough, especially considering many existing VAE variants (including those already discussed in the paper). \n\n2. One of the key results -- to generate molecules with similar property as the conditional property. Table 3 only compares with vanilla VAE, which doesn’t take condition property as input. The baseline is not persuasive. It will be more convincing if the authors can compare with other VAE types (i.e. include, but not limited to cVAE) and other non-VAE but reverse molecule design methods (i.e. GAN type, or not generative models).  Additionally, it is less convincing that the authors compare with different sets of existing methods on different tasks, for example Table 3, Table 4, Table 5. What are the results look like if say we add Table 4's other methods as a comparison for  table 3?  for example. JT-VAE (Jin et al., 2018)  , ORGAN (Guimaraes et al., 2017)  ,GCPN (You et al., 2018a) . The paper may need some more coherent experiments to be more persuasive. \n\n\n\n3. It will be more clear if the authors can mark the best values in tables in bold. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers agreed that the paper can be improved in several aspects such a motivation, novelty and framing of the contribution. The reviewers also liked the clarity of the presentation and some of the ideas. The reviewers constructive input may be used to improve this work. 
Decision: Reject