Original: L ATENT O PTIMIZATION V ARIATIONAL A UTOENCODER FOR C ONDITIONAL M OLECULAR G ENERATION
Revised: L ATENT O PTIMIZATION V ARIATIONAL A UTOENCODER FOR C ONDITIONAL M OLECULE G ENERATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Generative models such as variational autoencoder (VAE)  and generative adversarial networks are widely used for image, audio and text generation tasks.
Revised: Variational autoencoder (VAE) is a generation algorithm, consisting of an encoder and a decoder, and the latent variable is used as the input of the decoder. VAE is widely used for image, audio and text generation tasks.

Original: However, in general, the training of VAE isstill at risk of posterior collapsing especially for long sequential data.
Revised: In general, the training of VAE is at risk of posterior collapsing especially for long sequential data.

Original: To alleviate this, modiﬁed evidence lower bounds (ELBOs) were proposed.
Revised: To alleviate this, modiﬁed evidence lower bounds (ELBOs) were propsed.

Original: However, these approaches heuristically control training loss using a hyper-parameter, and it is not way to solve the fundamental problem of vanilla VAE.
Revised: However, these approaches heuristically control training loss using a hyper-parameter, and are not way to solve the fundamental problem of vanilla VAE.

Original: In this paper, we propose a latent optimization VAE (LOVAE) to insert an optimization step of the latent variable and alternately update the encoder and decoderof conditional VAE for maximizing ELBOs.
Revised: In this paper, we propose a method to insert an optimization step of the latent variable and alternately update the encoder and decoder for maximizing ELBOs.

Original: In experiments, we applied LOVAE on ZINC database, which contains 3D structural information of compound quality and molecular physical properties, for the  molecular generation.
Revised: In experiments, we applied the latent optimization VAE (LOVAE) on ZINC dataset, consisting of string representation of molecules, for the inverse molecular design.

Original: We showed that  LOVAE outperforms vanilla VAE in terms of ELBOs andthe molecular generation. In addition, LOVAE showed better performance in target property satisfaction and target property maximization tasks compared to existing works.
Revised: We showed that the proposed LOVAE is more stable in the training and achieves better performance than vanilla VAE in terms of ELBOs and molecular generation performance.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks (DNNs) have demonstrated a dramatic performance improvement in various applications.
Revised: Deep neural networks (DNNs) have demonstrated a dramatic performance improvement in various applications.

Original: Text extraction from image recognition, language translation, speech recognition, natural language recognition, and personal identiﬁcation by ﬁngerprint and iris have already achieved high accuracy (Wu et al., 2016;
Revised: Text extraction from image recognition, language translation, speech and natural language recognition, and personal identiﬁcation by ﬁngerprint and iris have already achieved high accuracy (Wu et al., 2016;

Original: Devlin et al., 2018; Awad, 2012; Nguyen et al., 2017).
Revised: Devlin et al., 2018; Awad, 2012; Nguyen et al., 2017).

Original: Recently, these applications became successful commercialized products.
Revised: Recently, these applications became successful commercialized products.

Original: For the purpose of generation of image, variational autoencoder (VAE) (Kingma & Welling, 2014), generative adversarial network (GAN) (Goodfellow et al., 2014), and reversible generative models (Kingma & Dhariwal, 2018) were proposed and showed much progress (Bleicher et al., 2003;
Revised: For the purpose of generation of image, variational autoencoder (VAE) (Kingma & Welling, 2014), generative adversarial network (GAN) (Goodfellow et al., 2014), and reversible generative models (Dinh et al., 2015; 2017; Kingma & Dhariwal, 2018) were proposed and showed much progress (Bleicher et al., 2003;

Original: Phatak et al., 2009; Grathwhohl et al., 2018).
Revised: Phatak et al., 2009; Grathwhohl et al., 2018).

Original: These generative models were initially studied in image data and showed better performance than previous models.
Revised: These generative models were initially studied in image data and showed better performance than previous models.

Original: Since then, it has been extended in research areas to generate new sentences (Iqbal & Qureshi, 2020) and to discover new drugs (Chen et al., 2018) and materials (Kim et al., 2018a).
Revised: Since then, it has been extended in research area to generate new sentences (Iqbal & Qureshi, 2020) and to discover new drugs (Chen et al., 2018) and materials (Kim et al., 2018a).

Original: Traditional materials research consists of four steps: molecule design, physical or chemical property prediction, molecular synthesis, and experimental evaluation.
Revised: Traditional materials research consists of four steps: molecule design, physical or chemical property prediction, molecular synthesis, and experimental evaluation.

Original: These steps are repeated until  desired molecular properties of a molecular structure are satisﬁed.
Revised: These steps are repeated until the desired molecular properties of a molecular structure are satisﬁed.

Original: Until now, trial-and-error techniques based on human knowledge have been widely used.
Revised: Until now, trial-and-error techniques based on human knowledge have been widely used.

Original: However, they are time consuming and very expensive.
Revised: However, they are time consuming and very expensive.

Original: In order to improve the traditional method, research for a high-throughput computational screening (HTCS) (Bleicher et al., 2003) was conducted.
Revised: In order to improve the traditional method, research for a high-throughput computational screening (HTCS) (Bleicher et al., 2003) was conducted.

Original: However, this also had limitations such as high computational cost, predeﬁned molecular structures by human knowledge, and low accuracy of simulation.
Revised: However, this also had limitations such as high computational cost, predeﬁned molecular structures by human knowledge, and low accuracy of simulation.

Original: Unlike the traditional approach, inverse molecular design is an attempt to ﬁnd novel molecules that satisfy desired properties from exploring a large chemical space (Sanchez-Lengeling & Aspuru-Guzik, 2018).
Revised: Unlike the traditional approach, inverse molecular design is an attempt to ﬁnd novel molecules that satisfy desired properties from exploring a large chemical space (Sanchez-Lengeling & Aspuru-Guzik, 2018).

Original: It extracts knowledge of potential molecular structures and properties from accumulated molecular structure databases (PubChem, ZINC, etc.) and proposes new molecular structures that do not exist in their database (Bolton et al., 2008; Irwin et al., 2012).
Revised: It extracts knowledge of potential molecular structures and properties from accumulated molecular structure databases (PubChem, ZINC, etc.) and proposes new molecular structures that do not exist in their database (Bolton et al., 2008; Irwin et al., 2012).

Original: With the inverse molecular design, it is possible to save cost by conducting molecular synthesis and experimental evaluation only for molecular structures having desired properties instead of searching almost inﬁnite chemical space.
Revised: With the inverse molecular design, it is possible to save cost by conducting molecular synthesis and experimental evaluation only for molecular structures having desired properties instead of searching almost inﬁnite chemical space.

Original: With the development of machine learning techniques, the generative models such as GAN and
Revised: With the development of machine learning techniques, the generative models such as GAN and

Original: VAE have been applied to inverse molecular design tasks in recent years (Sanchez-Lengeling &
Revised: VAE have been applied to inverse molecular design tasks in recent years (Sanchez-Lengeling &

Original: Aspuru-Guzik, 2018; Shi et al., 2020; Yang et al., 2020; Jin et al., 2018; Wang et al., 2020).
Revised: Aspuru-Guzik, 2018; Shi et al., 2020; Yang et al., 2020; Jin et al., 2018; Wang et al., 2020).

Original: In GAN, the discriminator tries to distinguish the molecular structure from the generator, and the generator tries to generate similar molecular structure from the database, simultaneously.
Revised: In GAN, the discriminator tries to distinguish the molecular structure from the generator, and the generator tries to generate similar molecular structure from the database, simultaneously.

Original: In VAE,the encoder outputs latent variables using the molecular structure as an input, andthe decoder generates the original molecular structure using the latent variables fromthe encoder.
Revised: In VAE, encoder outputs latent variables using the molecular structure as an input, and decoder generates the original molecular structure using the latent variables from encoder.

Original: However, there are difﬁculties in training generative models, such as poor convergence.
Revised: However, there is a difﬁculty in training generative models, such as poor convergence.

Original: In GAN, it is difﬁcult to learn due to the alternative update method from the two player game situation (Goodfellow et al., 2014).
Revised: In GAN, it is difﬁcult to learn due to the alternative update method from the two player game situation (Goodfellow et al., 2014).

Original: Problems such as the two models oscillation without adversarial, or the fact that the parameters are no longer learned in a speciﬁc situation due to the mode collapsing phenomenon, are still problems to be solved.
Revised: Problems such as the two models oscillation without adversarial, or the fact that the parameters are no longer learned in a speciﬁc situation due to the mode collapsing phenomenon, are still problems to be solved.

Original: In VAE, evidence lower bound (ELBO) for training VAE models requires training both the reconstruction and the KL-divergence loss of the latent variables, but it can cause a phenomenon called posterior collapsing.
Revised: In VAE, evidence lower bound (ELBO) for training VAE models requires training both the reconstruction and the KL-divergence loss of the latent variables, but it can cause a phenomenon called posterior collapsing.

Original: In order to prevent the posterior collapse, β -VAE (Higgins et al., 2017), Re-balancing-VAE (Yan et al., 2019), KL-annealing (Bowman et al., 2016), and controlVAE (Shao et al., 2020) have been proposed.
Revised: In order to prevent the posterior collapse, beta-VAE (Higgins et al., 2017), Re-balancing-VAE (Yan et al., 2019) and KL-annealing (Bowman et al., 2016) have been proposed.

Original: In this paper, we propose a latent optimization VAE (LOVAE) that provides stable learning method by inserting the latent variable optimization technique in conditional VAE (cVAE)
Revised: In this paper, we propose a latent optimization VAE (LOVAE) that provides stable learning method by inserting the latent variable optimization technique in conditional VAE (cVAE)

Original: (Sohn et al., 2015).
Revised: (Sohn et al., 2015).

Original: We apply two stages of the latent optimization to vanilla VAE.
Revised: We apply two stages of the latent optimization to vanilla VAE.

Original: By  training encoder ﬁrst, the latent variablefrom the encoder has been optimized in the direction of reducing the training loss in the same  data.
Revised: By ﬁrst training encoder once, the latent variable has been optimized in the direction of reducing the training loss in the same input data.

Original: After random sampling of the latent variables from the updated encoder, an additional latent optimizationbased gradient descent was applied.
Revised: After a reparameterization of the latent variables from the updated encoder, an additional latent optimization was applied.

Original: Our proposed method, LOVAE, was compared and veriﬁed in the inverse molecular design task, and a drug-like molecular structure (ZINC database (Sterling & Irwin, 2015)) was used as database.
Revised: Our proposed method, LOVAE, was compared and veriﬁed in the inverse molecular design task, and a drug-like molecular structure (ZINC dataset (Sterling & Irwin, 2015)) was used as database.

Original: We show the proposed method outperforms vanilla VAE in terms of reconstruction loss and ELBO, which are training indicators.
Revised: We show the proposed method outperform vanilla VAE in terms of reconstruction loss and ELBO, which are training indicators.

Original: In addition, it showed a more improved appearance inmolecular generation phase.
Revised: In addition, it showed a more improved appearance in generation phase.

Original: Consequently, generating molecules of LOVAE showed higher uniqueness ratio, novelty ratio, and target property satisfaction than some of previous approaches.
Revised: Consequently, generating molecules of LOVAE showed higher uniqueness, novelty ratio, and target property satisfaction than some of previous approaches.

Original: And, also, LOVAE generated molecules that showed higher value of penalized LogP property than the existing methods.
Revised: And, also, LOVAE generated molecules that showed higher value of penalized LogP property than the existing methods.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Inverse molecular design The inverse molecular design based on human-knowledge is very time consuming and relies on the intuition of the researcher, so many researchers recently tried to solve it through simulation method and others.
Revised: Inverse molecular design The inverse molecular design based on human-knowledge is very time consuming and relies on the intuition of the researcher, so many researchers recently tried to solve it through simulation method and others.

Original: HTCS (Bleicher et al., 2003), One of major simulation method, is an automated computational process that can rapidly identify active compounds, antibodies or genes, and the results provide starting points for many chemical works such as drug and material design (Chen et al., 2018;
Revised: HTCS (Bleicher et al., 2003), One of major simulation method, is an automated computational process that can rapidly identify active compounds, antibodies or genes, and the results provide starting points for many chemical works such as drug and material design (Chen et al., 2018;

Original: Kim et al., 2018a).
Revised: Kim et al., 2018a).

Original: HTCS uses a kind of brute-force method for searching and analyzing the desired chemical characteristics of molecule using a combination of hundreds or tens of millions of active compounds, but this can be disadvantage because the absolutely large amount of resources used to ﬁnd the desired goals (Phatak et al., 2009).
Revised: HTCS uses a kind of brute-force method for searching and analyzing the desired chemical characteristics of molecule using a combination of hundreds or tens of millions of active compounds, but this can be disadvantage because the absolutely large amount of resources used to ﬁnd the desired goals (Phatak et al., 2009).

Original: Recently, to solve the problems, artiﬁcial intelligence approaches have been widely applied in the ﬁeld of the molecular design under the name of the inverse molecular design.
Revised: Recently, to solve the problems, artiﬁcial intelligence approaches have been widely applied in the ﬁeld of the molecular design under the name of the inverse molecular design.

Original: The VAE and GAN  have been applying to the inverse molecular design ﬁeld (Sanchez-Lengeling &Aspuru-Guzik, 2018;
Revised: The VAE and GAN are typical generative models and they have been applying to the inverse molecular design ﬁeld (Sanchez-Lengeling &

Original:  Jin et al., 2020; Yang et al., 2020; Simm et al., 2020).
Revised: Aspuru-Guzik, 2018; Jin et al., 2020; Yang et al., 2020; Simm et al., 2020).

Original: For the inverse molecular design, various formats for simple representation of molecular structure information have been deﬁned instead of atom’s xyz coordinates.
Revised: For the inverse molecular design, various formats for simple representation of molecular structure information have been deﬁned instead of atom’s xyz coordinates.

Original: First, MDL format that represents 3D coordinate information and binding information between adjacent atoms together (Dalby et al., 1992).
Revised: First, MDL format that represents 3D coordinate information and binding information between adjacent atoms together.

Original: Extended Connectivity Fingerprint (ECFP) (Rogers & Hahn, 2010) and Simpliﬁed Molecular 
Revised: Extended Connectivity Fingerprint (ECFP) (Rogers & Hahn, 2010) and Simpliﬁed Molecular Input Line Entry

Original: Input Line Entry System (SMILES) (Weininger et al., 1989) are another representation for the molecular structure as a sequential character string.
Revised: System (SMILES) (Weininger et al., 1989) are another representation for the molecular structure as a sequential character string.

Original: And recently, the method of representing the molecular structure as a graph structure is also being researched.
Revised: And recently, the method of representing the molecular structure as a graph structure is also being researched.

Original: Among them, SMILES, string representation of molecules, is relatively easy to handle and has been showing a good performance.
Revised: Among them, SMILES, string representation of molecules, is relatively easy to handle and has been showing a good performance.

Original: Generative models
Revised: Generative models

Original: The generative models such as GAN and VAE are very sensitive to the latent variables.
Revised: The generative models such as GAN and VAE are very sensitive to the latent variables.

Revised: In order words, the training of latent variables greatly affects the performance of the generative models.

Original: In order words, according to ways of dealing with latent variables, difﬁcult problems have been occurred such as posterior collapsing and mode collapsing in the training.
Revised: However, according to way of dealing with latent variables, difﬁcult problems have been occurred such as posterior collapsing and mode collapsing in the training.

Original: As solutions for these phenomenons, methods to adjust a KL loss weight  have been proposed in VAE (Higgins et al., 2017;
Revised: As a solution for this phenomenon, methods to adjust a KL loss weight to a value other than 1 have been proposed in VAE.

Original: Bowman et al., 2016; Shao et al., 2020).

Original: In Kim et al.
Revised: In Kim et al.

Original: (2018b), latent variable from encoder is optimized according to maximizing of ELBO, and both of the encoder and decoder parameters are updated as the optimized latent variable.
Revised: (2018b), latent variable from encoder is optimized according to maximizing of ELBO, and both of the encoder and decoder parameters are updated as the optimized latent variable.

Original: In Zhang et al.
Revised: In Zhang et al.

Original: (2020), the reconstruction loss on the latent space is additive to the previous loss term.
Revised: (2020), the reconstruction loss on the latent space is additive to the previous loss term.

Original: In GAN, there are several works applying the latent variable optimization.
Revised: In GAN, there are several works applying the latent variable optimization.

Original: The method in Bojanowski et al.
Revised: The method in Bojanowski et al.

Original: (2017) optimizes the latent variable by assuming that the latent variable is learnable noise.
Revised: (2017) optimizes the latent variable by assuming that the latent variable is learnable noise.

Original: In Wu et al.
Revised: In Wu et al.

Original: (2019), randomly sampled latent variables are performed a gradient descent in the direction of reducing the loss of GAN.
Revised: (2019), randomly sampled latent variables are performed a gradient descent in the direction of reducing the loss of GAN.

Original: After that, the parameters of the discriminator and generator are updated with the loss from the optimized latent variable z .
Revised: After that, the parameters of the discriminator and generator are updated with the loss from the optimized latent variable z .

Original: All of these approaches resulted in a training stability and performance improvement in GAN.
Revised: All of these approaches resulted in a training stability and performance improvement in GAN.

Original:  C ONDITIONAL VAE( C VAE)
Revised:  C ONDITIONAL VAE

Original: In the cVAE (Kang & Cho, 2018; Kingma et al., 2014), the input variable x is assumed to be generated from a generative distribution p θ ( x | y , z ) conditioned on the output variable y and latent variable z .
Revised: In the conditional VAE (Kang & Cho, 2018; Kingma et al., 2014), the input variable x is assumed to be generated from a generative distribution p θ ( x | y , z ) conditioned on the output variable y and latent variable z .

Original: The prior distribution of z is assumed to be p ( z ) =
Revised: The prior distribution of z is assumed to be p ( z ) =

Original: N ( z | 0 , I ) .
Revised: N ( z | 0 , I ) .

Original: We use variational inference to approximate the posterior distribution of z given x and y by
Revised: We use variational inference to approximate the posterior distribution of z given x and y by

Original: [Equation]
Revised: [Equation]

Original: From the perspective of the auto-encoder, q φ ( z | x , y ) and p θ ( x | y , z ) are called as an encoder and a decoder, respectively.
Revised: From the perspective of the auto-encoder, q φ ( z | x , y ) and p θ ( x | y , z ) are called as an encoder and a decoder, respectively.

Original: Feed-forward neural networks are used for µ φ ( x , y ) and σ φ ( x , y ) .
Revised: Feed-forward neural networks are used for µ φ ( x , y ) and σ φ ( x , y ) .

Original: The objective of the cVAE is to maximize ELBO, which is a lower bound of the marginal loglikelihood:
Revised: The objective of the conditional VAE is to maximize ELBO, which is a lower bound of the marginal log-likelihood:

Original: [Equation]
Revised: [Equation]

Original: up to an additive constant, where KL denotes the Kullback-Leibler divergence.
Revised: up to an additive constant, where KL denotes the Kullback-Leibler divergence.

Original: Given a random sample z generated from the encoder q φ ( z | x , y ) , deﬁne the total loss as
Revised: Given a random sample z generated from the encoder q φ ( z | x , y ) , deﬁne the total loss as

Original: [Equation]
Revised: [Equation]

Original: Then, −L total ( x , y , z ) is Monte Carlo approximation of ELBO equation 2.
Revised: Then, −L total ( x , y , z ) is Monte Carlo approximation of ELBO equation 2.

Original: We deﬁne
Revised: We deﬁne

Original: L recon ( x , y , z ) = − log p θ ( x | y , z ) , because it can be regarded as a reconstruction loss.
Revised: L recon ( x , y , z ) = − log p θ ( x | y , z ) , because it can be regarded as a reconstruction loss.

Original: In vanilla cVAE, parameters θ and φ are jointly optimized to minimize L total .
Revised: In vanilla conditional VAE, parameters θ and φ are jointly optimized to minimize L total .

Original: A string representation of a molecule called SMILES is widely used to analyze molecular data (Higgins et al., 2017; Yan et al., 2019; Kang & Cho, 2018).
Revised: A string representation of a molecule called SMILES is widely used to analyze molecular data (Higgins et al., 2017; Yan et al., 2019; Kang & Cho, 2018).

Original: To deal with string data like SMILES in cVAE, recurrent neural networks (Yan et al., 2019; Kang & Cho, 2018) are used for the decoder p θ ( x | y , z ) .
Revised: To deal with string data like SMILES in conditional VAE, recurrent neural networks (Yan et al., 2019; Kang & Cho, 2018) are used for the decoder p θ ( x | y , z ) .

Original: Given a target molecular property y , a new molecule x having this property is generated in the following way:
Revised: Given a target molecular property y , a new molecule x having this property is generated in the following way:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: # Update the Encoder
Revised: # Update the Encoder

Original: Generate z ∼ q φ ( z | x , y ) .
Revised: Generate z ∼ q φ ( z | x , y ) .

Original: Calculate L total ( x , y , z ) .
Revised: Calculate L total ( x , y , z ) .

Original: Update φ (cid:48)
Revised: Update φ (cid:48)

Original: ← φ using L total ( x , y , z ) .
Revised: ← φ using L total ( x , y , z ) .

Original: # Additive Latent Optimization
Revised: # Additive Latent Optimization

Original: Generate z (cid:48) ∼ q φ (cid:48)
Revised: Generate z (cid:48) ∼ q φ (cid:48)

Original: ( z (cid:48)
Revised: ( z (cid:48)

Original: | x , y ) .
Revised: | x , y ) .

Original: Update z (cid:48)(cid:48)
Revised: Update z (cid:48)(cid:48)

Original: ← z (cid:48) − α β
Revised: ← z (cid:48) − α β

Original: + ||
Revised: + ||

Original: g
Revised: g

Original: || 2 g where g = ∂
Revised: || 2 g where g = ∂

Original: L recon ( x , y , z (cid:48) )
Revised: L recon ( x , y , z (cid:48) )

Original: ∂
Revised: ∂

Original: z (cid:48) .
Revised: z (cid:48) .

Original: # Update the Decoder
Revised: # Update the Decoder

Original: Calculate L recon ( x , y , z (cid:48)(cid:48) ) .
Revised: Calculate L recon ( x , y , z (cid:48)(cid:48) ) .

Original: Update θ (cid:48)
Revised: Update θ (cid:48)

Original: ← θ using L recon ( x , y , z (cid:48)(cid:48) ) .
Revised: ← θ using L recon ( x , y , z (cid:48)(cid:48) ) .

Original:  L ATENT O PTIMIZATION VAE(LOVAE)
Revised:  L ATENT O PTIMIZATION VAE

Original: Since the latent variable z generated from the encoder is inputted to the decoder to calculate L total , the encoder and decoder are closely related. If the encoder generates non-informative or highly volatile z , then the decoder is difﬁcult to reconstruct x from z .
Revised: At ﬁrst, we have considered advancing VAE in terms of a latent variable optimization. The problem we originally wanted to consider in this paper is that vanilla VAE training is not in an optimized process. The decoder is trained depending on the encoder output z .

Original: In this case, it maynot be optimal to update the encoder and decoder simultaneously.
Revised: However, in the vanilla VAE, the latent value, which is the result of encoder before the update with the same input data x , is used for the decoder training. From the perspective of the decoder, when the same input data x is used, it may be more effective to calculate L total by using the updated latent variable ( z (cid:48) ) passing through the updated encoder. Our proposed method, LOVAE, tried to solve this problem in terms of the latent variable optimization.

Original: The decoder is very sensitive to the encoderresults, z , and the accuracy of the decoder training may vary according to the encoder results.
Revised: LOVAE uses the same input x for learning the encoder and decoder, and ( z (cid:48) ) is used for updating the decoder because the encoder is updated ﬁrst.

Original: In other words, the result of the encoder can be optimized in advance in  direction  in which the decoder trains well.
Revised: From this approach, L total becomes smaller than the vanilla VAE ( L total ( x , y , z ) > L total ( x , y , z (cid:48) ) ). In addition, it helped the decoder training by optimizing ( z (cid:48) ) in the direction of reducing the l L total ( x , y , z (cid:48) ) one more in a way that does not spoil the training of encoder and decoder.

Original: That is, better encoder and better latent variable can make the decoder even better.
Revised: That is, better encoder and better latent variable can make the decoder even better.

Original: In this section, we propose LOVAE in which the encoder and decoder are alternatively trained.

Original: To be more speciﬁc, the encoder is updated as usual with the decoder while the decoder is ﬁxed, andan additive optimization of the latent variable from the encoder follows.
Revised: To be more speciﬁc, the encoder is updated as usual with the decoder while the decoder is ﬁxed, and optimization of the latent variable from the encoder follows.

Original: Finally, the decoder is updated with the optimized latent variable from the updated encoder.
Revised: Finally, the decoder is updated with the optimized latent variable.

Original: Updating encoder ﬁrst has an effect similar to latent optimization.
Revised: Updating encoder ﬁrst has an effect similar to latent optimization.

Original: By this updating, a more suitable z can be created and this latent variable not only reduces the loss but also depends on the input data x .
Revised: By this updating, a more suitable z can be created and this latent variable not only reduces the loss but also depends on the input data.

Original: A brief comparison of the vanilla  VAE and LOVAE is described in Figure 1.
Revised: A brief comparison of the vanilla (conditional) VAE and LOVAE is described in Figure 1.

Original: First, the encoder parameter φ is updated to φ (cid:48) in the direction of reducing the total loss L total ( x , y , z ) where z is generated using the current encoder parameter φ .
Revised: First, the encoder parameter φ is updated to φ (cid:48) in the direction of reducing the total loss L total ( x , y , z ) where z is generated using the current encoder parameter φ .

Original: Secondly, z (cid:48) is generated using the updated encoder parameter φ (cid:48) , and then z (cid:48) is updated to z (cid:48)(cid:48) in the direction of reducing the reconstruction loss L recon ( x , y , z (cid:48) ) using a single step of natural gradient descent method (Wu et al., 2019).
Revised: Secondly, z (cid:48) is generated using the updated encoder parameter φ (cid:48) , and then z (cid:48) is updated to z (cid:48)(cid:48) in the direction of reducing the reconstruction loss L recon ( x , y , z (cid:48) ) using the natural gradient descent method (Wu et al., 2019).

Original: Lastly, the decoder parameter θ is updated to θ (cid:48) in the direction of reducing the reconstruction loss L recon ( x , y , z (cid:48)(cid:48) ) using the optimized z (cid:48)(cid:48) .
Revised: Lastly, the decoder parameter θ is updated to θ (cid:48) in the direction of reducing the reconstruction loss L recon ( x , y , z (cid:48)(cid:48) ) using the optimized z (cid:48)(cid:48) .

Original: Since the optimized z (cid:48)(cid:48) is used in the update of the decoder parameter, LOVAE is expected to achieve bigger ELBO and show stable convergence.
Revised: Since the optimized z (cid:48)(cid:48) is used in the update of the decoder parameter, LOVAE is expected to achieve bigger ELBO and show stable convergence.

Original: This will be veriﬁed with numerical results in Section 5.
Revised: This will be veriﬁed with numerical results in Section 5.

Original: Note that latent optimization is only applied in the training, and the inference of LOVAE remains same with the vanilla one.
Revised: Note that latent optimization is only applied in the training, and the inference of LOVAE remains same with the vanilla one.

Original: In summary, the whole process of LOVAE is detailed in Algorithm 1.
Revised: In summary, the whole process of an update is detailed in Algorithm 1.

Original:  E XPERIMENTAL R ESULTS 5. E XPERIMENT S ETUP
Revised:  E XPERIMENTAL R ESULTS 5. E XPERIMENT S ETUP

Original: ZINC database (Sterling & Irwin, 2015) is a database that organizes information about various compounds drug-like molecules.
Revised: ZINC database (Sterling & Irwin, 2015) is a database that organizes information about various compounds drug-like molecules.

Original: ZINC contains 3D structural information of compound quality and molecular physical properties such as molecular weight (molwt), partition coefﬁcient (LogP), and quantitative estimation of drug-likeness (QED).
Revised: ZINC contains 3D structural information of compound quality and molecular physical properties such as molecular weight (molWt), partition coefﬁcient (LogP), and quantitative estimation of drug-likeness (QED).

Original: In addition, this information is provided in conformity with various molecular expression formats such as SMILES, mol2, and 3D SDF used in the chemical ﬁeld.
Revised: In addition, this information is provided in conformity with various molecular expression formats such as SMILES, mol2, and 3D SDF used in the chemical ﬁeld.

Original: SMILES is a string representation to express chemical substances, such as molecules, in the form of ASCII string.
Revised: SMILES is a string representation to express chemical substances, such as molecules, in the form of ASCII string.

Original: It is possible to express complex graph-structured chemicals in the sequence form using simple rules.
Revised: It is possible to express complex graph-structured chemicals in the sequence form using simple rules.

Original: Figure 2 showsan example molecule of SMILES  in ZINC DB, and we used two types of DB,
Revised: Figure 2 shows SMILES of a molecule in ZINC DB, and we used two types of DB,

Original: ZINC310K
Revised: ZINC310K

Original: (Yan et al., 2019; Kang & Cho, 2018) and ZINC250K (Kusner et al., 2017).
Revised: (Yan et al., 2019; Kang & Cho, 2018) and ZINC250K (Kusner et al., 2017).

Original: The vocabulary contains 39 different symbols {1, 2, 3, 4, 5, 6, 7, 8, 9, +, -, =, #, (, ), [, ], H, B, C, N, O, F, Si, P, S, Cl, Br, Sn, I, c, n, o, p, s, \ , /, @, @@}.
Revised: The vocabulary contains 39 different symbols {1, 2, 3, 4, 5, 6, 7, 8, 9, +, -, =, #, (, ), [, ], H, B, C, N, O, F, Si, P, S, Cl, Br, Sn, I, c, n, o, p, s, \ , /, @, @@}.

Original: The minimum, median, and maximum lengths of  SMILES strings of ZINC310K are 8, 42, and 86, respectively.
Revised: The minimum, median, and maximum lengths of a SMILES string of ZINC310K are 8, 42, and 86, respectively.

Original: (In case of ZINC250K, 9, 44, and 120)
Revised: (In case of ZINC250K, 9, 44, and 120)

Original: The distribution of target properties in our sampled database, ZINC310K, is shown in Figure 3.
Revised: The distribution of target properties in our sampled dataset, ZINC310K, is shown in Figure 3.

Original: Median values of molwt, LogP, and QED are 359.02, 2.91, and 0.70, respectively.
Revised: Median values of molwt, LogP, and QED are 359.02, 2.91, and 0.70, respectively.

Original: Among the existing VAE variants, β -VAE (Higgins et al., 2017), re-balancing VAE (Yan et al., 2019), 
Revised: Among the existing VAE variants, beta-VAE (Higgins et al., 2017), re-balancing VAE (Yan et al., 2019), and KL-annealing VAE

Original: KL-annealing VAE (Bowman et al., 2016), and controlVAE (Shao et al., 2020) control the weight ofL KL  in  equation 3 to achieve their own purpose, such as the disentanglement of latent variables, the avoidance of the posterior collapse, and the stability of the training.
Revised: (Bowman et al., 2016) control the weight of KL loss in the total loss equation 3 to achieve their own purpose, such as the disentanglement of latent variables, the avoidance of the posterior collapse, and the stability of the training.

Original: The proposed latent optimization technique is also applicable with those methods.
Revised: The proposed latent optimization technique is also applicable with those methods.

Original: Unlike LOVAE, semi-amortized VAE (SA-VAE) in Kim et al.
Revised: Unlike LOVAE, semi-amortized VAE (SA-VAE) in Kim et al.

Original: (2018b) optimizes latent variables without the encoder update phase and applied a momentum based optimization multiple times.
Revised: (2018b) updates latent variable without the encoder update phase and applied a momentum based optimization multiple times.

Original: After that, decoder and encoder parameters are updated according to the optimized latent variable.
Revised: After that, decoder and encoder parameters are updated according to the optimized latent variable.

Original: According to Kim et al. (2018b), SA-VAE utilized the encoder and latent optimization parts even in test phase.
Revised: According to (Kim et al., 2018b), SA-VAE utilized the encoder and latent optimization parts even in test phase.

Revised: That is, SA-VAE is not proper to our task, because it needs the latent optimization part.

Original: On the other hand, LOVAE does not utilize the additive latent optimizationand the encoder in the inference(molecular generation) phase. 
Revised: On the other hand, LOVAE does not utilize the additive latent optimization in the inference phase. Like LOVAE, there is an existing study that learns the encoder ﬁrst (He et al., 2019). In the case of

Revised: He et al.

Revised: (2019), the encoder is updated several times until certain condition is satisﬁed, and different input data are used when updating encoder and decoder.

Original: The basic model structure in this paper follows a general VAE structure for sequential data.
Revised: LOVAE differs in some ways. It updates the encoder ﬁrst, but updates it only once without any stop condition, and uses the same input data for the encoder and decoder, and performs the additive latent optimization with the reparameterization to help overall VAE learning. We think z (cid:48) obtained by the updating encoder at the same input data is more natural and suitable for the decoder learning. In addition, we think that using the same input data for the training encoder and decoder is more effective in terms of the latent optimization than using different input data to the encoder and decoder.

Revised: Also, Table 1 shows that LOVAE requires less training time.

Revised: We referred to the experimental results of Kim et al.

Revised: (2018b).

Revised: Although SA-VAE and

Revised: [Table]

Revised: [Table]

Revised: (He et al., 2019) used Yahoo dataset from Yang et al.

Revised: (2017), it was compared with the same method, vanilla VAE.

Revised: The basic model structure in this paper follows the general VAE structure for sequential data.

Original: The encoder has a bi-directional RNN structure, and the decoder is a uni-directional RNN structure (Yan et al., 2019; Kang & Cho, 2018).
Revised: The encoder has a bi-directional RNN structure, and the decoder is a uni-directional RNN structure (Yan et al., 2019; Kang & Cho, 2018).

Original: Each RNN structure was composed of three layers of GRU cells, and the dimension of the latent variable was set to 100.
Revised: Each RNN structure was composed of three layers of GRU cells, and the dimension of the latent variable was set to 100.

Original: The hidden size of each GRU cells is 250, and the dimension of the properties is 3.
Revised: The hidden size of each GRU cells is 250, and the dimension of the properties is 3.

Original: A 103-dimensional vector in which a 100-dimensional latent variable and a 3-dimensional property are concatenated is inputted to the decoder.
Revised: A 103-dimensional vector in which a 100-dimensional latent variable and a 3-dimensional property are concatenated is inputted to the decoder.

Original: Vanilla VAE and
Revised: Vanilla VAE and

Original: LOVAE have the exactly same model structure, but only training strategy differs.
Revised: LOVAE have the exactly same model structure, but only training strategy differs.

Original: We used the Adam optimizer with β 1 = 0 .
Revised: We used the Adam optimizer with β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: 999 , and (cid:15) =
Revised: 999 , and (cid:15) =

Original: 10 − 6 , and a polynomial-based learning rate decay was applied.
Revised: 10 − 6 , and a polynomial-based learning rate decay was applied.

Original: The initial and end learning rate for the learning rate decay were 0.001 and 0.0, respectively.
Revised: The initial and end learning rate for the learning rate decay were 0.001 and 0.0, respectively.

Revised: In the case of max epoch for the training, several cases were tried.

Revised: Among them, we set a number of the max epoch that shows good performance.

Original: During the training, we normalize each property value to have a meanand standard deviation 1.
Revised: During the training, we normalize each property value to have a mean 0 and standard deviation 1.

Original: For the additive latent optimization of LOVAE, α = 0 .
Revised: For the additive latent optimization of LOVAE, α = 0 .

Original: 001 and β = 5 were applied.
Revised: 001 and β = 5 were applied.

Original: 5. E VALUATION : T RAINING L OSS
Revised: 5. E VALUATION : VAE TRAINING PHASE

Original: As deﬁned in Section 3, L total of VAE is the sum of L recon and L KL .
Revised: As deﬁned in Section 3, L total of VAE is the sum of L recon and L KL .

Original: It can be thought that L total is primarily important, and the importance of L recon and L KL can be determined depending on the purpose.
Revised: It can be thought that L total is primarily important, and the importance of L recon and L KL can be determined depending on the purpose.

Original: For the evaluation, models initialized with 5 different random seeds were trained in each algorithm, vanilla VAE and LOVAE.
Revised: For the evaluation, models initialized with 5 different random seeds were trained in each algorithm, vanilla VAE and LOVAE.

Revised: The vanilla VAE is based on cVAE.

Revised: The trained models were compared and analyzed with L total and L recon of the train set.

Original: The loss of LOVAE was measured without the additive latent[Table] [Table] optimization.
Revised: The loss of LOVAE was measured without the additive latent optimization.

Original: Table 1 shows the training results for vanilla VAE and LOVAE.
Revised: Table 2 shows the training results for vanilla VAE and LOVAE.

Original: We tried each method ﬁve times and calculated the mean of the training losses.
Revised: We tried each method ﬁve times and calculated the mean of the training losses.

Original: ‘Original ELBO’ utilized equation 3 as the total loss [Table] and ‘reduced KL term’ reduced the weight of L KL of equation 3 like Yan et al.
Revised: ‘Original ELBO’ utilized equation 3 as the total loss, and ‘reduced KL term’ reduced the weight of L KL of equation 3 like Yan et al.

Original: In our experiments, when the weight of KL loss was from 0.7 to 0.8, each method showed a smaller L total .
Revised: In our experiments, when the weight of KL loss was from 0.7 to 0.8, each method showed a smaller L total .

Original: Figures in round brackets are losses of LOVAE without the additive latent optimization during training ( α = 0 . 0 )

Original: (to verify the updating encoder ﬁrst only).

Original: It can be seen that LOVAE is better than vanilla VAE in both L recon and L total .
Revised: It can be seen that LOVAE is better than vanilla VAE in both L recon and L total .

Original: In the results of ‘original ELBO’, L recon of LOVAE is on average 2.75 less than that of vanilla VAE.
Revised: In the results of ‘original ELBO’, L recon of LOVAE is on average 2.74 less than that of vanilla VAE.

Original: In terms of L KL , LOVAE is 1.67 greater, but L total of LOVAE is 1.08 smaller.
Revised: In terms of L KL , LOVAE is 1.68 greater, but L total of LOVAE is 1.07 smaller.

Original: In case of ‘reduced KL term’, LOVAE also shows a smaller L total .
Revised: In case of ‘reduced KL term’, LOVAE also shows a smaller L total .

Original: The relative improvement of  L total is about 4.45% in the case of ‘original ELBO’.
Revised: The relative improvement of t L total is about 4.45% in the case of ‘original ELBO’.

Original: LOVAE with α = 0 .outperformed vanilla VAE about 0.86 in terms of L total .

Original: 5. E VALUATION : T ARGET M OLECULAR G ENERATION
Revised: 5. E VALUATION : MOLECULAR GENERATION PHASE

Original: The generated molecules  can be evaluated according to three criteria before considering their property values.
Revised: The generated molecules from the generative model can be evaluated according to three criteria.

Original: The ﬁrst one is a validity.
Revised: The ﬁrst one is a validity.

Original: This means that the generated molecule has a sound structure.
Revised: This means that the generated molecule has a sound structure.

Original: This can be  determined using the RDKit package (Landrum).
Revised: This can be [Table] [Table] determined using the RDKit package (Landrum).

Original: The second one isa novelty.
Revised: The second one is novelty.

Original: The purpose of the inverse molecular design is to ﬁnd new molecules that have not discovered yet.
Revised: The purpose of the inverse molecular design is to ﬁnd new molecules that have not discovered yet.

Original: A molecule is said to be novel if it is not in the train database.
Revised: A molecule is said to be novel if it is not in the train set.

Original: The third one is an uniqueness which refers to how molecules are generated without overlapping.
Revised: The third one is uniqueness. If the latent space is very narrow and latent variable z is repeatedly sampled in a similar space, there is much room for generating the same SMILES.

Original: Generally, the more uniqueness, the better the generative model.
Revised: That is, the more uniqueness, the better the generative model.

Original: The generated molecules are more meaningful if they satisfy all three criteria, validity, novelty, and uniqueness.
Revised: The generated molecules are more meaningful if they satisfy all three criteria, validity, novelty, and uniqueness.

Original: In this paper, the ratio that satisﬁes all three criteria is deﬁned as a generative efﬁciency.
Revised: In this paper, the ratio that satisﬁes all three criteria is deﬁned as a generative efﬁciency.

Original: For example, if a generative model attempts to generate 1,000 molecules and has 600 molecules that satisfy validity, novelty, and uniqueness at once, the generative efﬁciency is 0.6.
Revised: For example, if a generative model attempts to generate 1,000 molecules and has 600 molecules that satisfy validity, novelty [Table] and uniqueness at once, the generative efﬁciency is 0.6.

Original: For evaluation, three values were determined for each property as a condition for the generative model, and ZINC310K was used.
Revised: For evaluation, three values were determined for each property as a condition for the generative model, and ZINC310K was used.

Original: It was determined to be close to the median, lower 10%, and upper 10% value in our train set.
Revised: It was determined to be close to the median, lower 10%, and upper 10% value in our train set.

Original: In case of molwt, 360.0, 260.0, and 460.0 were used as the condition.
Revised: In case of molwt, 360.0, 260.0, and 460.0 were used as the condition.

Original: For LogP and QED, {3.0, 1.5, 4.5} and {0.7, 0.5, 0.9} were chosen, respectively (Kang & Cho, 2018).
Revised: For LogP and QED, {3.0, 1.5, 4.5} and {0.7, 0.5, 0.9} were chosen, respectively.

Original: Among three target properties, the other two except one of the target property were sampled from a conditional prior distribution given the target value.

Original: The molecular generation was attempted 3,500 times in each condition value, and 31,500 molecules were generated in a total of 9 conditions.
Revised: The molecular generation was attempted 3,500 times in each condition value, and 31,500 molecules were generated in a total of 9 conditions.

Original: For the analysis, ‘original ELBO’ models were used.
Revised: For the analysis, ‘original ELBO’ models were used.

Original: The results of the generative efﬁciency are shown in Table 2.
Revised: The results of the generative efﬁciency are shown in Table 3.

Original: It can be seen that LOVAE has goodperformance of the generative efﬁciency  in all properties.[Table] [Table] [Table]
Revised: It can be seen that LOVAE has good generative efﬁciency and uniform performance in all properties.

Original:  E VALUATION : T ARGET P ROPERTY S ATISFACTION
Revised: 5. E VALUATION : PROPERTY SATISFACTION

Revised: In addition to the generative efﬁciency, it is possible to use property satisfaction as a measure of performance for cVAE.

Original: A target property satisfaction is how many molecules with properties being close to the target condition can be generated.
Revised: It is how many molecules with properties being close to the target condition can be generated.

Original: For this, evaluation of property satisfaction was conducted based on two criteria.
Revised: For this, evaluation of property satisfaction was conducted based on two criteria.

Original: The ﬁrst is the percentage that the property value of the generated molecule falls within 10% of the error range of the condition value.
Revised: The ﬁrst is the percentage that the property value of the generated molecule falls within 10% of the error range of the condition value.

Original: For example, if the target value of QED is 0.7, it measures the percentage of the generated molecules whose property value lies between 0.63 and 0.77.
Revised: For example, if the target value of molwt is 360.0, it measures the percentage of the generated molecules whose property value lies between 324.0 and 396.0.

Original: The results of the experiment are shown in Table 3.
Revised: The results of the experiment are shown in Table 4.

Original: The target values and the number of generation are the same to Sec. 5.

Original: [Table]

Original: and each property of generated molecules was computed using RDkit.

Original: In all three properties, LOVAE showed higher property satisfaction.
Revised: In all three properties, LOVAE showed higher property satisfaction.

Original: At 10% and 5% error property satisfaction, LOVAE showed a relatively 12.1% and 10.5% improvement, respectively.
Revised: At 10% and 5% error property satisfaction, LOVAE showed a relatively 12.1% and 10.5% improvement, respectively.

Original: For the comparison with previous works using ZINC250K, we referred to You et al.
Revised: For the comparison with previous works using ZINC250K, we referred to You et al.

Original: (2018b).
Revised: (2018b).

Original: In You et al. (2018b), a property targeting task was performed, and speciﬁc ranges of molwt and LogP were considered.
Revised: In that paper, property targeting task was performed, and speciﬁc ranges of molwt and LogP were considered.

Original: In our case, if the target range of molwt is from 150 to 200, we conditioned LOVAE as 175.
Revised: In our case, if the target range of molWt is from 150 to 200, we conditioned LOVAE as 175.

Original: There are four target ranges  like Table 4.
Revised: The target ranges are four like Table 5.

Original: Except for the target range -2.5 (cid:53) LogP (cid:53) -2.0, LOVAE showed the best performance.
Revised: Except for the target range -2.5 (cid:53) LogP (cid:53) -2.0, LOVAE showed the best performance.

Original: Since percentages of the training data in the range of -2.5 (cid:53) LogP (cid:53) -2.0 and 5.0 (cid:53) LogP (cid:53) 5.5 are 0.28% and 1.30%,respectively, the ﬁrst target range, -2.5 (cid:53) LogP (cid:53) -2.0, can be a bit more difﬁcult than 5.0 (cid:53) LogP (cid:53) 5.5 for cVAE structures.
Revised: Since percentages of the training data in the range of -2.5 (cid:53) LogP (cid:53) -2.0 and 5 (cid:53) LogP (cid:53) 5.5 are 0.28% and 1.30%, the ﬁrst target range can be a bit more difﬁcult.

Original: In this respect, the result of LOVAE at the target range -2.5 (cid:53) LogP (cid:53)
Revised: In this respect, the result of LOVAE at the target range -2.5 (cid:53) LogP (cid:53)

Original: -2.0 seems to make sense.
Revised: -2.0 seems to make sense.

Original: [Table]
Revised: [Table]

Original: molecularRNN (Popova et al., 2019) 10. graphAF (Shi et al., 2020) 12. AllSMILES (Nigam et al., 2020) 16. LOVAE 20.
Revised: [Table]

Original: LOVAE with plogP15.
Revised: LOVAE with plogP

Original: [Figure]
Revised: [Table]

Revised: 5. E VALUATION : PROPERTY MAXIMIZATION ( PENALIZED L OG P AND QED)

Original: In the many previous papers, a property maximization task was performed and it was evaluated on penalized LogP
Revised: In the many previous papers, a property maximization task was performed and it was evaluated on penalized LogP

Original: (pLogP) and QED (Kusner et al., 2017).
Revised: (pLogP) and QED (Kusner et al., 2017).

Original: plogP has an unbounded range, while QED has a  range of [0, 1] by deﬁnition.
Revised: QED is a property with a boundary range with [0, 1], but the range of penalized logP is ( −∞ , ∞ ) .

Original: pLogP is a LogP penalized by the synthetic accessibility score (SA) and the number of large rings (cycle), pLogP = logP - SA - cycle, and it can be calculated by RDKit.
Revised: pLogP is a LogP penalized by the synthetic accessibility score (SA) and the number of large rings (cycle), pLogP = logP - SA - cycle.

Original: It can be thought as an extrapolation task because generative models have to create a new molecule that is not in the range of the property values of the training database (The maximum pLogP in ZINC250K = 5.072).
Revised: It can be thought as a extrapolation task because generative models have to create a new molecule that is not in the range of the property values of the training DB (The maximum pLogP in training DB = 5.072).

Original: In order to ﬁnd new molecules which show the highest property, some of previous approaches utilized a reward function or a property regressors with Bayesian optimization (Kusner et al., 2017; Shi et al., 2020; Gómez-Bombarelli et al., 2018).
Revised: In order to ﬁnd new molecules which show the highest property, some of previous approaches utilized a reward function or a property regressors with sparse Gaussian process (Kusner et al., 2017; Shi et al., 2020; Gómez-Bombarelli et al., 2018).

Original: LOVAE, which is cVAE structure, is not very suitable for this task. In addition, the maximizing pLogP task is a little less important because it generates ‘unusual’ molecules with high probability like long aliphatic chains (Winter et al., 2019; Zhou et al., 2019). Despite the above reasons, for comparison with the existing works, we just conditionedLOVAE by a high value such as 30.0 and 0.98 for LogP and QED, respectively, and 20k of molecular generation were tried.
Revised: In our approach, LOVAE, we just conditioned by a high value such as 30.0 and 0.98 for LogP and QED, respectively.

Original: Table 5 shows the property maximization results using ZINC250K. LOVAE generated the new molecules shown sufﬁciently high pLogP and QED properties.
Revised: Table 6 shows the property maximization results. LOVAE generated the new molecules shown the highest penalized LogP and QED property.

Original: It is noteworthy that LOVAE showed good performance only with LogP condition without an additive part like the rewardfunction or 
Revised: It is noteworthy that LOVAE showed good performance only with LogP condition without a separate part like a reward or property regressor.

Original: Bayesian optimization on latent space.

Original: In other words, LOVAE, which is a cVAE structure, works properly even in the extrapolation task.
Revised: That is, it was conﬁrmed that LOVAE, which is a conditional VAE type, works properly even in the extrapolation task.

Original: The top 3 molecules of each property are representedin Fig.
Revised: Top 3 molecules of each property are represented Fig.

Original: In addition, LOVAE with “pLogP" as condition was trained and veriﬁed.
Revised: In addition, LOVAE with pLogP as condition was trained and veriﬁed.

Original: As the conditions of cVAE changed from [LogP, molwt, QED] to [pLogP, SA, cycle], it seems that LOVAE with pLogP learned a different z space from LOVAE.

Original: The performance was worse than LOVAE (with LogP), but since the number of large ringsand SA can also be given as condition, the trend of the generated molecules was different (Fig. 5). As a result, LOVAE with pLogP generated ‘usual’ molecules, such as not including long aliphatic chains.
Revised: The performance was worse than LOVAE with LogP, but since the number of large rings can also be given as condition, the trend of the generated molecules was different (5).

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, LOVAE applying latent optimization technique toconditional VAE algorithm is proposed.
Revised: In this paper, LOVAE applying latent optimization technique to VAE algorithm is proposed.

Original: By training the encoder ﬁrst, the latent variable z has an updated distribution in the direction of reducing the train loss.
Revised: By training the encoder ﬁrst, the latent variable z has an updated distribution in the direction of reducing the train loss.

Original: Additionally,a reparameterized z is updated  in the direction of reducing the reconstruction loss before training the decoder.
Revised: Additionally, z is updated before training the decoder in the direction of reducing the reconstruction loss.

Original: The training of decoder becomes more efﬁcient by utilizing our optimized z .
Revised: The training of decoder becomes more efﬁcient by utilizing the optimized z .

Original: This was applied and veriﬁed in the inverse molecular design task in ZINC database, and conﬁrmed that it showed a better appearance in the train loss, ELBO, and molecular generative performance than those of vanilla VAE and previous works.
Revised: This was applied and veriﬁed in the inverse molecular design task in ZINC dataset and conﬁrmed that it showed a better appearance in the train loss, ELBO, and molecular generative performance than those of vanilla VAE.