Original: M OLECULE O PTIMIZATION BY E XPLAINABLE E VOLUTION
Revised: M OLECULE O PTIMIZATION BY E XPLAINABLE E VOLUTION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Optimizing molecules for desired properties is a fundamental yet challenging task in chemistry, material science and drug discovery.
Revised: Optimizing molecules for desired properties is a fundamental yet challenging task in chemistry, material science and drug discovery.

Original: In this paper, we develop a novel algorithm for optimizing molecule properties via an Expectation Maximization (EM)-like explainable evolutionary process.
Revised: In this paper, we develop a novel algorithm for optimizing molecule properties via an Expectation Maximization (EM)-like explainable evolutionary process.

Original: Our algorithm is designed to mimic human experts in the process of searching for desirable molecules and alternate between two stages: the ﬁrst stage on explainable local search which identiﬁes rationales, i.e. critical subgraph patterns accounting for desired molecular properties, and the second stage on molecule completion which explores the larger space of molecules containing good rationales.
Revised: Our algorithm is designed to mimic human experts in the process of searching for desirable molecules and alternate between two stages: the ﬁrst stage on explainable local search which identiﬁes rationales, i.e. critical subgraph patterns accounting for desired molecular properties, and the second stage on molecule completion which explores the larger space of molecules containing good rationales.

Original: We test our method against various baselines on a real-world multi-property optimization task where each method is given the same number of queries to the property oracle.
Revised: We test our method against various baselines on a real-world multi-property optimization task where each method is given the same number of queries to the property oracle.

Original: We show that our evolution-by-explanation algorithm is 79% better than the best baseline in terms of a generic metric combining aspects such as success rate, novelty and diversity.
Revised: We show that our evolution-by-explanation algorithm is 79% better than the best baseline in terms of a generic metric combining aspects such as success rate, novelty and diversity.

Original: Human expert evaluation on optimized molecules shows that 60% of top molecules obtained from our methods are deemed as successful ones.
Revised: Human expert evaluation on optimized molecules shows that 60% of top molecules obtained from our methods are deemed as successful ones.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The space of organic molecules is vast, the size of which is exceeding 10 60 (Reymond et al., 2010).
Revised: The space of organic molecules is vast, the size of which is exceeding 10 60 (Reymond et al., 2010).

Original: Searching over this vast space for molecules of interest is a challenging task in chemistry, material science and drug discovery, especially given that molecules are desired to meet multiple criteria, e.g. , high potency and low toxicity in drug discovery.
Revised: Searching over this vast space for molecules of interest is a challenging task in chemistry, material science and drug discovery, especially given that molecules are desired to meet multiple criteria, e.g. , high potency and low toxicity in drug discovery.

Original: When human experts are optimizing molecules for better molecular properties, they will ﬁrst come up with rationales within desirable molecules.
Revised: When human experts are optimizing molecules for better molecular properties, they will ﬁrst come up with rationales within desirable molecules.

Original: Typically, the rationales are subgraphs in a molecule that are deemed contributing mostly to certain desired molecular properties.
Revised: Typically, the rationales are subgraphs in a molecule that are deemed contributing mostly to certain desired molecular properties.

Original: Once rationales are identiﬁed, chemists will design new molecules on top of rationales in the hope that, the desired properties of new molecules will be further enhanced due to the existence of rationale and changes of non-rationale parts.
Revised: Once rationales are identiﬁed, chemists will design new molecules on top of rationales in the hope that, the desired properties of new molecules will be further enhanced due to the existence of rationale and changes of non-rationale parts.

Original: The cycle of identifying molecular rationales and redesigning new hypothetical molecules will be carried on until molecules that meet certain property criteria are discovered.
Revised: The cycle of identifying molecular rationales and redesigning new hypothetical molecules will be carried on until molecules that meet certain property criteria are discovered.

Original: In this paper, we develop a novel algorithm that mimics the process of molecule optimization by human experts.
Revised: In this paper, we develop a novel algorithm that mimics the process of molecule optimization by human experts.

Original: Our algorithm ﬁnds new molecules with better properties via an EM-like explainable evolutionary process (Figure 1).
Revised: Our algorithm ﬁnds new molecules with better properties via an EM-like explainable evolutionary process (Figure 1).

Original: The algorithm alternates between two stages.
Revised: The algorithm alternates between two stages.

Original: During the ﬁrst stage, we use an explainable local search method to identify rationales within high-quality molecules that account for their high property scores.
Revised: During the ﬁrst stage, we use an explainable local search method to identify rationales within high-quality molecules that account for their high property scores.

Original: During the second stage, we use a conditional generative model to explore the larger space of molecules containing useful rationales.
Revised: During the second stage, we use a conditional generative model to explore the larger space of molecules containing useful rationales.

Original: Our method is novel in that we are using explainable models to help us exploit useful patterns in the molecules, yet leveraging generative models to help us explore the molecule landscape.
Revised: Our method is novel in that we are using explainable models to help us exploit useful patterns in the molecules, yet leveraging generative models to help us explore the molecule landscape.

Original: Comparing to existing methods which directly learn a generative model using Reinforcement Learning or perform continuous optimization in the latent space of molecules (Olivecrona et al., 2017; You et al., 2018a; Dai et al., 2018b), our method is more sample-efﬁcient and is able to generate more novel and unique molecules that meet the criteria.
Revised: Comparing to existing methods which directly learn a generative model using Reinforcement Learning or perform continuous optimization in the latent space of molecules (Olivecrona et al., 2017; You et al., 2018a; Dai et al., 2018b), our method is more sample-efﬁcient and is able to generate more novel and unique molecules that meet the criteria.

Original: We evaluate our algorithm against several state-of-the-art methods on a molecule optimization task involving multiple properties.
Revised: We evaluate our algorithm against several state-of-the-art methods on a molecule optimization task involving multiple properties.

Original: Compared with baselines, our algorithm is able to increase the success rate by 50%, novelty by 14%, while having a competitive diversity.
Revised: Compared with baselines, our algorithm is able to increase the success rate by 50%, novelty by 14%, while having a competitive diversity.

Original: We further propose a new metric, QNU score, to jointly consider all three aspects, and show that we achieve a score of 52.7% compared with 29.5% by the best baseline.
Revised: We further propose a new metric, QNU score, to jointly consider all three aspects, and show that we achieve a score of 52.7% compared with 29.5% by the best baseline.

Original: We also ask experienced chemists to evaluate topgenerated molecules and ﬁnd that 30 of them are as good as existing ones.
Revised: We also ask experienced chemists to evaluate topgenerated molecules and ﬁnd that 30 of them are as good as existing ones.

Original: [Figure]
Revised: [Figure]

Original: The main contributions of this paper are summarized below:
Revised: The main contributions of this paper are summarized below:

Original: • We propose a novel EM-like evolution-by-explanation algorithm for molecule optimization; • We present a novel, principled, explainable graph model based on an information-theoretic approach to extract subgraphs that are essential for maintaining certain desired properties; • Our approach outperforms existing state-of-the-arts by a large margin in terms of success rate (50% better), novelty (14% better), and an overall metric (79% better) on a real-world multiproperty optimization task.
Revised: • We propose a novel EM-like evolution-by-explanation algorithm for molecule optimization; • We present a novel, principled, explainable graph model based on an information-theoretic approach to extract subgraphs that are essential for maintaining certain desired properties; • Our approach outperforms existing state-of-the-arts by a large margin in terms of success rate (50% better), novelty (14% better), and an overall metric (79% better) on a real-world multiproperty optimization task.

Original: Related Work.
Revised: In recent years, there has been a surge of interest in using machine learning to discover novel molecules with certain properties.

Original: Most  existing work on molecule optimization deﬁnes a generative model for either the SMILES strings (Weininger, 1988) or molecular graphs, and use reinforcement learning to optimize the properties of  generated molecules (Olivecrona et al., 2017; Guimaraes et al., 2017; 
Revised: Most of the existing work deﬁne a generative model for either the SMILES strings (Weininger, 1988) or molecular graphs, and use Reinforcement Learning algorithms to optimize the properties of the generated molecules (Segler et al., 2018; Olivecrona et al., 2017; Guimaraes et al., 2017; You et al., 2018a;

Original: You et al., 2018a; Popova et al., 2018; 2019; Samanta et al., 2019;
Revised: Popova et al., 2018; 2019; Samanta et al., 2019;

Original: Zhou et al., 2019; De Cao & Kipf, 2018; Kearnes et al., 2019; Shi et al., 2020; Jin et al., 2020).
Revised: Zhou et al., 2019; De Cao & Kipf, 2018; Kearnes et al., 2019; Shi et al., 2020; Jin et al., 2020).

Revised: Others optimize the continuous representation of molecules in a latent space learned by variants of variational autoencoders (Kusner et al., 2017; Dai et al., 2018b; Jin et al., 2018; G´omez-Bombarelli et al., 2018; Kang & Cho, 2018; Liu et al., 2018; Kajino, 2019).

Revised: More recent work try Evolutionary algorithms (Nigam et al., 2020; Leguy et al., 2020; Winter et al., 2019), or focus on ﬁnding highquality molecules with synthesis paths (Bradshaw et al., 2019; Korovina et al., 2020; Gottipati et al., 2020).

Original: Most similar to our approach is RationaleRL (Jin et al., 2020), which extracts subgraphs from seed molecules using Monte Carlo Tree Search (MCTS) and generates full molecules by completing the subgraphs.
Revised: Most similar to our approach is RationaleRL (Jin et al., 2020), which extracts subgraphs from seed molecules using Monte Carlo Tree Search (MCTS) and generates full molecules by completing the subgraphs.

Original: Compared with previous work, our approach is the ﬁrst to incorporate an explainable model in the iterative search process.
Revised: Compared with previous work, our approach is the ﬁrst to incorporate an explainable model in the iterative search process.

Original: Similar work on explainable models  deﬁnes input pattern selection operators, such that the outputs of the model based on the selected input patterns have large mutual information with the original model outputs (Chen et al., 2018;
Revised: Existing work on explainable models approach the problems from three directions. The ﬁrst line of work use gradients of the outputs with respect to inputs to identify the salient features in the inputs (Simonyan et al., 2013; Springenberg et al., 2014; Baehrens et al., 2010); The second line of work approximate the model with simple interpretable models, such as locally additive models (Bach et al., 2015; Kindermans et al., 2016; Ribeiro et al., 2016; Lundberg & Lee, 2017; Shrikumar et al., 2017); the third line of work deﬁnes input pattern selection operators, such that the outputs of the model based on the selected input patterns has large mutual informaton with the original model outputs (Chen et al., 2018;

Original: Ying et al., 2019).
Revised: Ying et al., 2019).

Original: Our explainable model is different from
Revised: Our explainable model is different from

Original: GNNExplainer (Ying et al., 2019) in that we optimize the discrete subgraph structure with learned variational predictor, instead of directly feeding continuous edge masking into the target model.
Revised: GNNExplainer (Ying et al., 2019) in that we optimize the discrete subgraph structure with learned variational predictor, instead of directly feeding continuous edge masking into the target model.

Original: See Appendix A.1 for more discussions on both topics.

Original:  P ROBLEM S ETTING
Revised:  P ROBLEM S ETTING

Original: In this paper, we study the problem of discovering molecules g from the molecular space G with a high property score, measured by a scoring function f .
Revised: In this paper, we study the problem of discovering molecules g from the molecular space G with a high property score, measured by a scoring function f .

Original: And usually, there is a set of seed molecules
Revised: And usually, there is a set of seed molecules

Original: G 0 ⊂ G from experts with high scores to start with.
Revised: G 0 ⊂ G from experts with high scores to start with.

Original: More formally, the problem can be stated as
Revised: More formally, the problem can be stated as

Original: Molecule Optimization.
Revised: Molecule Optimization.

Original: Given a scoring function f : G (cid:55)→ [0 , 1] , and a set of seed molecules G 0 ⊂ G , the goal is to learn a molecule generative model p ( g ) such that the expected score of the generated molecules is maximized, i.e.,
Revised: Given a scoring function f : G (cid:55)→ [0 , 1] , and a set of seed molecules G 0 ⊂ G , the goal is to learn a molecule generative model p ( g ) such that the expected score of the generated molecules is maximized, i.e.,

Original: [Equation]
Revised: [Equation]

Original: To prevent the model p ( g ) from generating a small set of ﬁxed molecules with high scores, we additionally require the learned distribution to be both novel and diverse , i.e. able to generate molecules that are dissimilar to the set of reference molecules (a subset of G 0 ) and to each other.
Revised: To prevent the model p ( g ) from generating a small set of ﬁxed molecules with high scores, we additionally require the learned distribution to be both novel and diverse , i.e. able to generate molecules that are dissimilar to the set of reference molecules (a subset of G 0 ) and to each other.

Original: The molecule optimization problem in Eq (1) is combinatorial in nature, which poses a signiﬁcant challenge.
Revised: The molecule optimization problem in Eq (1) is combinatorial in nature, which poses a signiﬁcant challenge.

Original: To mimic the scientiﬁc discovery process, we allow the algorithm to query f on new molecules under a querying budget.
Revised: To mimic the scientiﬁc discovery process, we allow the algorithm to query f on new molecules under a querying budget.

Original: Examples of some well-known scoring functions include the QED score measuring the drug-likeness (Bickerton et al., 2012), SA score measuring the synthetic accessibility (Ertl & Schuffenhauer, 2009), TPSA score measuring the ability to permeate cells (Prasanna & Doerksen, 2009), etc.
Revised: Examples of some well-known scoring functions include the QED score measuring the drug-likeness (Bickerton et al., 2012), SA score measuring the synthetic accessibility (Ertl & Schuffenhauer, 2009), TPSA score measuring the ability to permeate cells (Prasanna & Doerksen, 2009), etc.

Original: The scoring function is general and could also encode multi-property objectives (Olivecrona et al., 2017; Brown et al., 2019).
Revised: The scoring function is general and could also encode multi-property objectives (Olivecrona et al., 2017; Brown et al., 2019).

Original: Optimizing multiple properties together suffers from the sparsity of high scores, a scenario which is shown to be more challenging than single property optimization (Jin et al., 2020).
Revised: Optimizing multiple properties together suffers from the sparsity of high scores, a scenario which is shown to be more challenging than single property optimization (Jin et al., 2020).

Original: When experts are optimizing the molecular property, they will ﬁrst look for substructures that result in the formation of that property, and use them as the foundation for building novel molecules.
Revised: When experts are optimizing the molecular property, they will ﬁrst look for substructures that result in the formation of that property, and use them as the foundation for building novel molecules.

Original: These subgraphs are called rationales (examples in Figure 1).
Revised: These subgraphs are called rationales (examples in Figure 1).

Original: The set of rationales is formally deﬁned as,
Revised: The set of rationales is formally deﬁned as,

Original: S = { s | ∃ g ∈ G , s .
Revised: [Equation] [Equation]

Original: t . s is a subgraph ofg } .
Revised: subgraph of

Original: [Equation]
Revised: [Equation]

Original:  O UR F RAMEWORK
Revised:  O UR F RAMEWORK

Original: Our novel framework for optimizing molecular property with generative models consists of a modeling component and an algorithm component.
Revised: Our novel framework for optimizing molecular property with generative models consists of a modeling component and an algorithm component.

Original: In our modeling component, we propose a rationalebased hierarchical generative model for p ( g ) , which ﬁrst generate rationales and then complete molecules.
Revised: In our modeling component, we propose a rationalebased hierarchical generative model for p ( g ) , which ﬁrst generate rationales and then complete molecules.

Original: In our algorithm component, we design an alternating optimization procedure that interleaves between rationale distribution optimization and molecule generative model optimization.
Revised: In our algorithm component, we design an alternating optimization procedure that interleaves between rationale distribution optimization and molecule generative model optimization.

Original: Furthermore, we develop a novel explainable graph model to effectively carry out the rationale model optimization.
Revised: Furthermore, we develop a novel explainable graph model to effectively carry out the rationale model optimization.

Original: Next, we will ﬁrst start describing our hierarchical generative model.
Revised: Next, we will ﬁrst start describing our hierarchical generative model.

Original: 3. R ATIONALE - BASED H IERARCHICAL G ENERATIVE M ODEL
Revised: 4. R ATIONALE - BASED H IERARCHICAL G ENERATIVE M ODEL

Original: To tackle the challenging search problem, we develop a hierarchical generative model that mimics the process of molecule optimization by human experts.
Revised: To tackle the challenging search problem, we develop a hierarchical generative model that mimics the process of molecule optimization by human experts.

Original: In our model, we ﬁrst sample rationales s from a distribution p ( s ) , and then molecules g will be generated according to conditional distribution p θ ( g | s ) .
Revised: In our model, we ﬁrst sample rationales s from a distribution p ( s ) , and then molecules g will be generated according to conditional distribution p θ ( g | s ) .

Original: More speciﬁcally, our overall molecular generative model p θ ( g ) can be deﬁned as
Revised: More speciﬁcally, our overall molecular generative model p θ ( g ) can be deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where θ is the parameter of the conditional generative model, p ( s ) is the latent rationales distribution.
Revised: where θ is the parameter of the conditional generative model, p ( s ) is the latent rationales distribution.

Original: Here p θ ( g | s ) is a graph completion model from rationale s .
Revised: Here p θ ( g | s ) is a graph completion model from rationale s .

Original: The architecture of p θ ( g | s ) can be arbitrary.
Revised: The architecture of p θ ( g | s ) can be arbitrary.

Original: In this work, we use a latent variable model with a Gaussian prior p ( z ) ,
Revised: In this work, we use a latent variable model with a Gaussian prior p ( z ) ,

Original: [Equation]
Revised: [Equation]

Original: where p θ ( g | s, z ) is a variant of the GraphRNN (You et al., 2018b; Liu et al., 2018) by conditioning the graph generation on subgraphs.
Revised: where p θ ( g | s, z ) is a variant of the GraphRNN (You et al., 2018b; Liu et al., 2018) by conditioning the graph generation on subgraphs.

Original: As part of the initialization, p θ ( g | s ) is ﬁrst pretrained on
Revised: As part of the initialization, p θ ( g | s ) is ﬁrst pretrained on

Original: ChEMBL (Gaulton et al., 2017), a drug-like molecule dataset, in the same fashion as the variational autoencoder (Kingma & Welling, 2013), where the encoder is a standard GCN with atoms as vertices and bonds as edges.
Revised: ChEMBL (Gaulton et al., 2017), a drug-like molecule dataset, in the same fashion as the variational autoencoder (Kingma & Welling, 2013), where the encoder is a standard GCN with atoms as vertices and bonds as edges.

Original: Note that different from p ( z ) , which is a ﬁxed prior, p ( s ) will be updated in each round.
Revised: Note that different from p ( z ) , which is a ﬁxed prior, p ( s ) will be updated in each round.

Original: And since representing a distribution on S is difﬁcult, we will use particles to represent p ( s ) in the algorithm.
Revised: And since representing a distribution on S is difﬁcult, we will use particles to represent p ( s ) in the algorithm.

Original: In order to improve the diversity of the generated molecules, we will also regularize the entropy of the rationale distribution p ( s ) , leading to the following diversity promoting objective function
Revised: In order to improve the diversity of the generated molecules, we will also regularize the entropy of the rationale distribution p ( s ) , leading to the following diversity promoting objective function

Original: [Equation]
Revised: [Equation]

Original: with a hyperparameter λ > 0 controlling the strength of the regularization.
Revised: with a hyperparameter λ > 0 controlling the strength of the regularization.

Original: 3. A LTERNATING O PTIMIZATION A LGORITHM
Revised: 4. A LTERNATING O PTIMIZATION A LGORITHM

Original: As the rationales and the molecules are coupled in the molecular graph space, directly optimizing the diversity promoting objective in Eq (5) would be very challenging.
Revised: As the rationales and the molecules are coupled in the molecular graph space, directly optimizing the diversity promoting objective in Eq (5) would be very challenging.

Original: Therefore we seek to optimize p θ ( g | s ) and p ( s ) in an alternating fashion, akin to the Expectation-Maximization (EM) algorithm.
Revised: Therefore we seek to optimize p θ ( g | s ) and p ( s ) in an alternating fashion, akin to the Expectation-Maximization (EM) algorithm.

Original: That is, the algorithm alternates between two stages:
Revised: That is, the algorithm alternates between two stages:

Original: • Expectation step (E-step) for obtaining an updated distribution p ( s ) , and • Maximization step (M-step) for improving the molecule completion model p θ ( g | s ) .
Revised: • Expectation step (E-step) for obtaining an updated distribution p ( s ) , and • Maximization step (M-step) for improving the molecule completion model p θ ( g | s ) .

Original: We name this algorithm MolEvol (Algorithm 1) by making an analogy to evolving a group of molecules over time (Figure 1).
Revised: We name this algorithm MolEvol (Algorithm 1) by making an analogy to evolving a group of molecules over time (Figure 1).

Original: Assume that, at iteration t − 1 , we already have p θ t − 1 ( g | s ) and p t − 1 ( s ) , and the set of seed samples G t − 1 drawn from p θ t − 1 ( g | s ) .
Revised: Assume that, at iteration t − 1 , we already have p θ t − 1 ( g | s ) and p t − 1 ( s ) , and the set of seed samples G t − 1 drawn from p θ t − 1 ( g | s ) .

Original: Then, at iteration t , we have,
Revised: Then, at iteration t , we have,

Original: E-
Revised: E-

Original: step.
Revised: step.

Original: We want to maximize the objective J with respect to the latent distribution p ( s ) given p θ t − 1 ( g | s ) .
Revised: We want to maximize the objective J with respect to the latent distribution p ( s ) given p θ t − 1 ( g | s ) .

Original: That is
Revised: That is

Original: [Equation]
Revised: [Equation]

Original: which is a maximum entropy estimation problem.
Revised: which is a maximum entropy estimation problem.

Original: Interestingly, the solution of the above optimization problem can be obtained in close form.
Revised: Interestingly, the solution of the above optimization problem can be obtained in close form.

Original: [Equation]
Revised: [Equation]

Original: where Z θ is a normalizing constant.
Revised: where Z θ is a normalizing constant.

Original: This updated distribution for the latent rationales will be needed for the later M-step.
Revised: This updated distribution for the latent rationales will be needed for the later M-step.

Original: However, since directly integrating with respect to p t ( s ) is difﬁcult, we will leverage sampling strategies and obtain m particles { s i } mi =1 from this distribution for later use in
Revised: However, since directly integrating with respect to p t ( s ) is difﬁcult, we will leverage sampling strategies and obtain m particles { s i } m i =1 from this distribution for later use in

Original: M-step.
Revised: M-step.

Original: However, computing the normalizing constant Z θ is difﬁcult, making direct sampling from p t ( s ) not straightforward.
Revised: However, computing the normalizing constant Z θ is difﬁcult, making direct sampling from p t ( s ) not straightforward.

Original: Standard sampling algorithms like Markov Chain Monte Carlo (Andrieu et al., 2003) could be extremely slow due to the lack of a good proposal distribution and the absence of gradients in the discrete graph space.
Revised: Standard sampling algorithms like Markov Chain Monte Carlo (Andrieu et al., 2003) could be extremely slow due to the lack of a good proposal distribution and the absence of gradients in the discrete graph space.

Original: To address this challenge, we will maintain a ﬁnite support set S t as the proposal, which is obtained from an explainable graph model (more details in the next section).
Revised: To address this challenge, we will maintain a ﬁnite support set S t as the proposal, which is obtained from an explainable graph model (more details in the next section).

Original: More speciﬁcally, suppose the explainable graph model, Explain ( · ) : G (cid:55)→ S , can take a graph input g and output the corresponding rationale s which explains why the graph g can obtain a high property score according to f ( g ) .
Revised: More speciﬁcally, suppose the explainable graph model, Explain ( · ) : G (cid:55)→ S , can take a graph input g and output the corresponding rationale s which explains why the graph g can obtain a high property score according to f ( g ) .

Original: Then support set S t can be maintained as follows
Revised: Then support set S t can be maintained as follows

Original: [Equation]
Revised: [Equation]

Original: where G 0 is provided to the algorithm initially by experts.
Revised: where G 0 is provided to the algorithm initially by experts.

Original: The rationales s ∈ S t will be treated as the set of particle locations for representing p t ( s ) .
Revised: The rationales s ∈ S t will be treated as the set of particle locations for representing p t ( s ) .

Original: Furthermore, for each of these particle locations, we will compute its unnormalized probability according to p t ( s ) in Eq (7), and then re-sample a set of m particles, { s i } mi =1 , as the ﬁnal representation for p t ( s ) (Andrieu et al., 2003).
Revised: Furthermore, for each of these particle locations, we will compute its unnormalized probability according to p t ( s ) in Eq (7), and then re-sample a set of m particles, { s i } m i =1 , as the ﬁnal representation for p t ( s ) (Andrieu et al., 2003).

Original: M-step.
Revised: M-step.

Original: With { s i } mi =1 from p t ( s ) , the Monte Carlo estimate of the objective function in Eq (5) becomes
Revised: With { s i } mi =1 from p t ( s ) , the Monte Carlo estimate of the objective function in Eq (5) becomes

Original: [Equation]
Revised: [Equation]

Original: We can then maximize it with respect to the parameters θ using REINFORCE, 
Revised: We can then maximize it with respect to the parameters θ using REINFORCE, [Equation] t

Original: [Equation]
Revised: [Equation]

Original: where α > 0 , g i ∼ p θ t − 1 ( ·| s i) .
Revised: where α > 0 , g i ∼ p [Equation] ( ·| s i

Original: [Equation]
Revised: [Equation]

Original: After the parameter is updated to θ t , we will sample a seed set of molecules G t from p θ t ( g | s ) by completing the rationale samples { s i } mi =1 using the updated model.
Revised: After the parameter is updated to θ t , we will sample a seed set of molecules G t from p θ t ( g | s ) by completing the rationale samples { s i } mi =1 using the updated model.

Original: That is
Revised: That is

Original: G t = { g i } n s i =1 , where g i ∼ g θ t ( ·| s ) , s ∼ Uniform ( { s 1 , s 2 , . .
Revised: [Equation] where g i ∼ g θ t ( ·| s ) , s ∼ Uniform ( { s 1 , s 2 , . .

Original: , s m } ) .
Revised: , s m } ) .

Revised: [Equation]

Original: The overall algorithm can be found in Algorithm 1.
Revised: The overall algorithm is summarized in Algorithm 1.

Original: p ( s ) and p θ ( g | s ) are updated in the E-step (line 3-4) and M-step (line 5-8), respectively.
Revised: p ( s ) and p θ ( g | s ) are updated in the E-step (line 3-4) and M-step (line 5-8), respectively.

Revised: A discussion on its convergence can be found in

Original:  Input: Seed molecules G 0 , pretrained graph completion model p θ ( g | s ) on ChEMBL.
Revised: Appendix A.4. Input: Seed molecules G 0 , pretrained graph completion model p θ ( g | s ) on ChEMBL.

Original: 1 Initialize S[-
-]
Revised: 1 Initialize S[-
-]

Original: = {} .

Original: 2 for t ← 1 to N rounds do
Revised: 2 for t ← 1 to N rounds do

Original: S t =
Revised: S t =

Original: S t − 1 ∪ { Explain ( g ) : g ∈ G t − 1 } .
Revised: S t − 1 ∪ { Explain ( g ) : g ∈ G t − 1 } .

Original: Sample s 1 , s 2 , · · · ,s m fromS t usingEq (7) with self-normalization.
Revised: Sample s 1 , s 2 , · · · , m from using (7) with self normalization.

Original: for j← 1 to N epochs do
Revised: for j to N epochs do

Original: Sample g1 , · · · , g m from p θ ( g | s 1 ) , · · · , p θ ( g | s m ) respectively. Update θ with REINFORCE
Revised: Sample g

Original: (Eq (10)). Sample seed molecules G t with Eq (11).
Revised: , g m from

Original: 9 return p θ ( g )MLP
Revised: ) , · · · , p θ s m )

Original: 3. E XPLAINABLE G RAPH M ODEL FOR R ATIONALES
Revised: 4. E XPLAINABLE G RAPH M ODEL FOR R ATIONALES

Original: In the E-step, it is crucial to update the support set for the rationales, such that the particles can be placed in spaces where p t ( s ) is large.
Revised: In the E-step, it is crucial to update the support set for the rationales, such that the particles can be placed in spaces where p t ( s ) is large.

Original: As we optimize p θ t ( g | s ) , this model can generate molecules with improved property scores.
Revised: As we optimize p θ t ( g | s ) , this model can generate molecules with improved property scores.

Original: Intuitively, we would also like to have an increasingly “good” support set for the rationales.
Revised: Intuitively, we would also like to have an increasingly “good” support set for the rationales.

Original: To do this, we will identify substructures in the current seed set of molecules G t which can best explain their high property scores, and add these discovered substructures as new rationales.
Revised: To do this, we will identify substructures in the current seed set of molecules G t which can best explain their high property scores, and add these discovered substructures as new rationales.

Original: Furthermore, we can measure the goodness of these substructure using their mutual information with the property value, and optimize the selector for these substructures using a variational formulation (Chen et al., 2018).
Revised: Furthermore, we can measure the goodness of these substructure using their mutual information with the property value, and optimize the selector for these substructures using a variational formulation (Chen et al., 2018).

Original: This entire procedure is illustrated in Figure 2 can also be seen as seeking explanations why molecules have high property scores.
Revised: This entire procedure is illustrated in Figure 2 can also be seen as seeking explanations why molecules have high property scores.

Original: Explainer.
Revised: Explainer.

Original: A molecular graph g is represented by g =
Revised: A molecular graph g is represented by g =

Original: ( V g , E g ) with atoms V g as vertices and bonds
Revised: ( V g , E g ) with atoms V g as vertices and bonds

Original: E g as edges.
Revised: E g as edges.

Original: For any subset U ⊆
Revised: For any subset U ⊆

Original: V g , the induced subgraph s
Revised: V g , the induced subgraph s

Original: =
Revised: =

Original: ( U , E U g ) is a subgraph of g formed by the vertices U and the edges
Revised: ( U , E U g ) is a subgraph of g formed by the vertices U and the edges

Original: E U g
Revised: E U g

Original: = { e ∈ E g | e start , e end ∈ U} connecting pairs of vertices in the subset.
Revised: = { e ∈ E g | e start , e end ∈ U} connecting pairs of vertices in the subset.

Original: An explainer Explain ( · ) : G (cid:55)→ S takes a graph g as an input, and outputs an induced subgraph s of k vertices.
Revised: An explainer Explain ( · ) : G (cid:55)→ S takes a graph g as an input, and outputs an induced subgraph s of k vertices.

Original: Variational Objective.
Revised: Variational Objective.

Original: We want to learn an explainer for the conditional distribution P ( Y = 1 | g ) (cid:44) f
Revised: We want to learn an explainer for the conditional distribution P ( Y = 1 | g ) (cid:44) f

Original: ( g ) (treating f ( g ) as a probability), with random variables Y ∈ { 0 , 1 } where
Revised: ( g ) (treating f ( g ) as a probability), with random variables Y ∈ { 0 , 1 } where

Original: Y = 1 indicates that the molecule has the property, and
Revised: Y = 1 indicates that the molecule has the property, and

Original: 0 otherwise.
Revised: 0 otherwise.

Original: We will learn a graph vertex sampler h φ ( g ) jointly with a variational approximation Q ( Y | g ) of P ( Y | g ) , such that the mutual information between Y and s is maximized
Revised: We will learn a graph vertex sampler h φ ( g ) jointly with a variational approximation Q ( Y | g ) of P ( Y | g ) , such that the mutual information between Y and s is maximized

Original: [Equation]
Revised: [Equation]

Original: such that s =
Revised: such that s =

Original: ( U , E U g ) and U ∼ h φ ( g ) . 
Revised: ( U , E U g ) and U ∼ h φ ( g ) . [Equation]

Original:  Details on sampling U from h φ ( g ) are presented in the next paragraph.
Revised: [Figure] Details on sampling U from h φ ( g ) are presented in the next paragraph.

Original: After sampling U , we can construct an induced subgraph s =
Revised: After sampling U , we can construct an induced subgraph s =

Original: ( U , E U g ) .
Revised: ( U , E U g ) .

Original: During the explanation, we then perform an additional expanding step,s (cid:48) =
Revised: During the explanation, we then perform an additional expanding step,

Original: ( U (cid:48) , E U (cid:48) g ) , where U (cid:48)
Revised: [Equation] [Equation] U (cid:48)

Original: =
Revised: =

Original: U ∪ { v |∃ v ∈ U , s . t .
Revised: U ∪ { v |∃ v ∈ U , s . t .

Original: e ( u, v ) ∈
Revised: e ( u, v ) ∈

Original: E g or u, v share a Benzene. } , (13) to obtain s (cid:48) , which deﬁnes the mapping s (cid:48)
Revised: E g or u, v share a Benzene [Equation] [Equation] 13) to obtain s (cid:48) , which deﬁnes the mapping s (cid:48)

Original: =
Revised: =

Original: Explain ( g ) (Algorithm 2).
Revised: Explain ( g ) (Algorithm 2).

Original: Parameterization of h φ ( g ) .
Revised: Parameterization of h φ ( g ) .

Original: Sampling subgraph s from g is equivalent to sampling a size- k subset
Revised: Sampling subgraph s from g is equivalent to sampling a size- k subset

Original: U from the vertices V g .
Revised: U from the vertices V g .

Original: We use a GNN h φ to deﬁne a vertex sampling policy
Revised: We use a GNN h φ to deﬁne a vertex sampling policy

Original: h φ ( g ) ∈ ∆ |V g | over the space of g ’s vertices.
Revised: h φ ( g ) ∈ ∆ |V g | over the space of g ’s vertices.

Original: Speciﬁcally, h φ ( g ) consists of two parts:
Revised: Speciﬁcally, h φ ( g ) consists of two parts:

Original: A message passing network (MPN) which outputs a matrix X g =
Revised: A message passing network (MPN) which outputs a matrix X g =

Original: MPN φ ( g ) ∈ R |V g |× d representing the d -dimensional embeddings for each vertex; 2.
Revised: MPN φ ( g ) ∈ R |V g |× d representing the d -dimensional embeddings for each vertex; 2.

Original: A fully-connected layer (FC) followed by a softmax layer to implement the vertex sampling policy h φ ( g ) =
Revised: A fully-connected layer (FC) followed by a softmax layer to implement the vertex sampling policy h φ ( g ) =

Original: Softmax ( FC φ ( X g )) ∈
Revised: Softmax ( FC φ ( X g )) ∈

Original: ∆ |V g | .
Revised: ∆ |V g | .

Original: Then we follow the procedure in L2X (Chen et al., 2018) for sampling k vertices one-by-one from the distribution h φ ( g ) using the Gumbel-softmax trick.
Revised: Then we follow the procedure in L2X (Chen et al., 2018) for sampling k vertices one-by-one from the distribution h φ ( g ) using the Gumbel-softmax trick.

Original: The sampled feature matrix can be written as X s = V ( φ, ζ ) (cid:12) X g ∈ R
Revised: The sampled feature matrix can be written as X s = V ( φ, ζ ) (cid:12) X g ∈ R

Original: |V g |× d , where ζ is a collection of auxiliary random variables sampled independently from the Gumbel distribution, V ( φ, ζ ) ∈ { 0 , 1 } |V g | is a mask on the rows of X g , and (cid:12) is the element-wise product.
Revised: |V g |× d , where ζ is a collection of auxiliary random variables sampled independently from the Gumbel distribution, V ( φ, ζ ) ∈ { 0 , 1 } |V g | is a mask on the rows of X g , and (cid:12) is the element-wise product.

Original: Parameterization of Q .
Revised: Parameterization of Q .

Original: Since directly using generic choices of Q to perform the variational approximation is hard, we approximate it with a MLP q ψ that takes the aggregated masked embedding vector x s = (cid:80)
Revised: Since directly using generic choices of Q to perform the variational approximation is hard, we approximate it with a MLP q ψ that takes the aggregated masked embedding vector x s = (cid:80)

Original: r X s [ r, :] ∈ R d as input, and predicts the target via Q ( Y = 1 | s ) =
Revised: r X s [ r, :] ∈ R d as input, and predicts the target via Q ( Y = 1 | s ) =

Original: q ψ ( X s ) =
Revised: q ψ ( X s ) =

Original: Sigmoid ( MLP ψ ( x s )) .
Revised: Sigmoid ( MLP ψ ( x s )) .

Original: Final Objective for Training.
Revised: Final Objective for Training.

Original: After applying the Gumbel-softmax trick, we transform the variational objective in Eq (12) into:
Revised: After applying the Gumbel-softmax trick, we transform the variational objective in Eq (12) into:

Original: [Equation]
Revised: [Equation]

Original: We can then apply stochastic gradient ascent to jointly optimize φ and ψ by sampling molecule g from the dataset and ζ from the Gumbel distribution.
Revised: We can then apply stochastic gradient ascent to jointly optimize φ and ψ by sampling molecule g from the dataset and ζ from the Gumbel distribution.

Original: We note that our design of the explainer model and the learning method is very different from those in GNNExplainer (Ying et al., 2019), which may be of independent interest in terms of explainable models for GNNs.
Revised: We note that our design of the explainer model and the learning method is very different from those in GNNExplainer (Ying et al., 2019), which may be of independent interest in terms of explainable models for GNNs.

Original: For instance, our explainable model h φ by itself is a GNN model, and we introduce a variational distribution q ψ which is optimized jointly with h φ .Algorithm
Revised: For instance, our explainable model h φ by itself is a GNN model, and we introduce a variational distribution q ψ which is optimized jointly with h φ .

Original: Input: Molecule g , vertex sampling policy φ .
Revised: Input: Molecule g , vertex sampling policy φ .

Original: h φ ( g ) =
Revised: [Equation]

Original: Softmax ( FC φ ( MPN φ ( g ))) . SampleU ∼ h φ ( g ) with Gumbel-softmax trick. s (cid:48)
Revised: Sample

Original: = Expand (( U , E U g )) as deﬁned in Eq (13). return s (cid:48)
Revised: Gumbel

Original: Rationale Extraction as Explaining.
Revised: Rationale Extraction as Explaining.

Original: During the E-step in our Algorithm 1, we utilize the trained explainer Explain ( · ) to extract rationales candidates s from the seed molecules.
Revised: During the E-step in our Algorithm 1, we utilize the trained explainer Explain ( · ) to extract rationales candidates s from the seed molecules.

Original: Then the candidates with the top Q -scores are added to the rationale support set to update S t . 
Revised: Then the candidates with the top Q -scores are added to the rationale support set to update S t . [Table]

Revised: Remark on Explanation.

Revised: In this paper we use the word “explanation” to refer to a critical component of the input that are of most importance for the ﬁnal prediction, following the convention of

Original: E XPERIMENTS
Revised: L2X (Chen et al., 2018) and GNNExplainer (Ying et al., 2019). However [Table] a more rigorous explanation using scientiﬁc language is rather important and helpful for scientiﬁc research.

Revised: Generating such explanation using a machine learning model could be highly relevant in general

Original:  We evaluate MolEvol on a multi-property molecule optimization task (Li et al., 2018; Jin et al., 2020) involving four properties:
Revised: [Table] which is beyond the scope of this paper.  E XPERIMENTS We evaluate MolEvol on a multi-property molecule optimization task (Li et al., 2018; Jin et al., 2020) involving four properties:

Original: • GSK-3 β : inhibition levels against glycogen synthase kinase-3 beta 
Revised: • GSK-3 β : inhibition levels against glycogen synthase kinase-3 beta (Li et al., 2018); • JNK3: inhibition levels against c-Jun N-terminal kinase 3 (Li et al., 2018); • QED: quantitative estimate of drug-likeness (Bickerton

Original: (Li et al.,2018)
Revised: et al.,

Original: ; • JNK3: inhibition levels against c-Jun N-terminal kinase 3 (Li et al., 2018); • QED: quantitative estimate of drug-likeness (Bickerton et al., 2012); • SA: synthetic accessibility (Ertl & Schuffenhauer, 2009).
Revised: ); • SA: synthetic accessibility (Ertl & Schuffenhauer, 2009).

Original: GSK-3 β and JNK3 are potential targets in the treatment of Alzheimer’s disease.
Revised: GSK-3 β and JNK3 are potential targets in the treatment of Alzheimer’s disease.

Original: Their corresponding property predictors are random forests trained on real-world experimental data using Morgan ﬁngerprint features (Rogers & Hahn, 2010).
Revised: Their corresponding property predictors are random forests trained on real-world experimental data using Morgan ﬁngerprint features (Rogers & Hahn, 2010).

Original: In our experiment, we consider all properties by combining their scores into a uniﬁed scoring function 1 :
Revised: In our experiment, we consider all properties by combining their scores into a uniﬁed scoring function 1 :

Original: [Equation]
Revised: [Equation]

Original: [Table] Note that in the eMolecules dataset (eMolecules, 2020) of commercially available molecules, only 0.007% out of over 27M molecules meet the criteria with f ( g ) > 0 . 5 .
Revised: Note that in the eMolecules dataset (eMolecules, 2020) of commercially available molecules, only 0.007% out of over 27M molecules meet the criteria with f ( g ) > 0 . 5 .

Original: Experiment Setting.
Revised: Experiment Setting.

Original: We provide a set of 3.4K seed molecules for the algorithms to start with.
Revised: We provide a set of 3.4K seed molecules for the algorithms to start with.

Original: Each seed molecule has a high value in GSK-3 β or JNK3 or both.
Revised: Each seed molecule has a high value in GSK-3 β or JNK3 or both.

Original: There is a budget on both the time and the number of queries.
Revised: There is a budget on both the time and the number of queries.

Original: Each algorithm is allowed to query f -scores no more than 5 M times and to run no more than 1 day on a Ubuntu 16.04.6 LTS server with 1 Nvidia RTX 2080 Ti GPU, and 20 Intel(R) Xeon(R) E5-2678 2.50GHz CPUs.
Revised: Each algorithm is allowed to query f -scores no more than 5 M times and to run no more than 1 day on a Ubuntu 16.04.6 LTS server with 1 Nvidia RTX 2080 Ti GPU, and 20 Intel(R) Xeon(R) E5-2678 2.50GHz CPUs.

Original: We evaluate the algorithms on 20K generated molecules using the following metrics.
Revised: We evaluate the algorithms on 20K generated molecules using the following metrics.

Original: We call a molecule g qualiﬁed if f ( g ) > 0 .
Revised: We call a molecule g qualiﬁed if f ( g ) > 0 .

Original: 5 , novel if the distance between g and the reference molecule set is larger than a threshold 2 .
Revised: 5 , novel if the distance between g and the reference molecule set is larger than a threshold 2 .

Original: The reference set contains 315 qualiﬁed molecules, which is a subset of the provided seed molecules.
Revised: The reference set contains 315 qualiﬁed molecules, which is a subset of the provided seed molecules.

Original: • Success rate: the percentage of qualiﬁed molecules out of 20K molecules.
Revised: • Success rate: the percentage of qualiﬁed molecules out of 20K molecules.

Original: • Novelty: the percentage of novel molecules out of all qualiﬁed molecules.
Revised: • Novelty: the percentage of novel molecules out of all qualiﬁed molecules.

Original: • Diversity: the average pairwise distance between all qualiﬁed and novel molecules 2 .
Revised: • Diversity: the average pairwise distance between all qualiﬁed and novel molecules 2 .

Original: • QNU score: the percentage of qualiﬁed , novel and unique molecules out of 20K molecules.
Revised: • QNU score: the percentage of qualiﬁed , novel and unique molecules out of 20K molecules.

Original: Success rate, novelty and diversity have been adopted as evaluation metrics in previous work (Olivecrona et al., 2017; Li et al., 2018; Jin et al., 2020).
Revised: Success rate, novelty and diversity have been adopted as evaluation metrics in previous work (Olivecrona et al., 2017; Li et al., 2018; Jin et al., 2020).

Original: However, the trade-off among the three targets complicates the comparisons between algorithms.
Revised: However, the trade-off among the three targets complicates the comparisons between algorithms.

Original: Therefore we propose a new metric, QNU score, to jointly consider the three aspects.
Revised: Therefore we propose a new metric, QNU score, to jointly consider the three aspects.

Original: QNU will serve as the major factor for comparison.
Revised: QNU will serve as the major factor for comparison.

Revised: [Figure]

Original: Baselines.
Revised: Baselines.

Original: We compare MolEvol against state-of-the-art molecule optimization algorithms below:
Revised: We compare MolEvol against state-of-the-art molecule optimization algorithms below:

Original: • RationaleRL (Jin et al., 2020) learns a graph completion model, but relies on a ﬁxed set of multi
Revised: • RationaleRL (Jin et al., 2020) learns a graph completion model, but relies on a ﬁxed set of multiproperty rationales composed by single-property rationales extracted by MCTS.

Original: - property

Original: rationales

Original: composed

Original: by single-property rationales

Original: extracted by MCTS Concretely each state in MCTS represents a subgraph of the molecule and the reward function is deﬁned as the property score of the subgraph.
Revised: Concretely, each state in MCTS represents a subgraph of the molecule and the reward function is deﬁned as the property score of the subgraph.

Original: • REINVENT (Olivecrona et al., 2017) learns a generative model for SMILES string with RL; • MSO (Winter et al., 2019) optimizes the property using Particle Swarm Optimization (PSO) (Kennedy & Eberhart, 1995) in a continuous latent space of molecules.
Revised: • REINVENT (Olivecrona et al., 2017) learns a generative model for SMILES string with RL; • MSO (Winter et al., 2019) optimizes the property using Particle Swarm Optimization (PSO) (Kennedy & Eberhart, 1995) in a continuous latent space of molecules.

Original: • GA-D(t) (Nigam et al., 2020) employs a genetic algorithm enhanced with a neural network.
Revised: • GA-D(t) (Nigam et al., 2020) employs a genetic algorithm enhanced with a neural network.

Original: Since MSO and GA-D(t) do not explicitly learn a generative model, we use the best 20K out of 5M molecules encountered in the search process for comparison.
Revised: Since MSO and GA-D(t) do not explicitly learn a generative model, we use the best 20K out of 5M molecules encountered in the search process for comparison.

Original: Results.
Revised: Results.

Original: The results are reported in Table 1.
Revised: The results are reported in Table 1.

Original: Comparing to the baselines, MolEvol achieves higher success rate in generating qualiﬁed molecules (30% higher than RationaleRL, MSO and GA-D(t), 45% higher than REINVENT).
Revised: Comparing to the baselines, MolEvol achieves higher success rate in generating qualiﬁed molecules (30% higher than RationaleRL, MSO and GA-D(t), 45% higher than REINVENT).

Original: Meanwhile, MolEvol maintains high novelty (75.7%) which may beneﬁt from the alternating process in the framework.
Revised: Meanwhile, MolEvol maintains high novelty (75.7%) which may beneﬁt from the alternating process in the framework.

Original: Although the diversity is slightly lower than
Revised: Although the diversity is slightly lower than

Original: RationaleRL due to the distribution shift during optimization, the QNU score, which takes all the above metrics into consideration, is signiﬁcantly higher than rationaleRL (52.7% versus 29.5%). 
Revised: RationaleRL due to the distribution shift during optimization, the QNU score, which takes all the above metrics into consideration, is signiﬁcantly higher than RationaleRL (52.7% versus 29.5%). Please refer to Appendix A.5 for additional discussions on other baselines.

Original: Ablation Studies.
Revised: Ablation Studies.

Original: We introduce baselines below to understand the importance of each component:
Revised: We introduce baselines below to understand the importance of each component:

Original: • [MCTS] replaces the explainable local search with MCTS as in Jin et al
Revised: • [MCTS] replaces the explainable local search with MCTS as in Jin et al.

Original: ); • [FixR] uses a ﬁxed set of rationales, i
Revised: ; • [FixR] uses a ﬁxed set of rationales, i

Original: e. only having one round of explainable local search; • [FixM] uses a ﬁxed (pretrained) model, i.e. having no molecule completion stage.[Equation] [Figure]
Revised: e. only having one round of explainable local search; • [FixM] uses a ﬁxed (pretrained) model, i.e. having no molecule completion stage.

Original: As illustrated in Table 1, MolEvol achieves the highest QNU score among all variants.
Revised: As illustrated in Table 1, MolEvol achieves the highest QNU score among all variants.

Original: The large performance gap (success rate: 93.0% vs. 67.3%/66.3%; QNU 7% vs ) between MolEvol and [FixM]/[FixR] justiﬁes the necessity of taking both E-step and M-step into consideration.
Revised: The large performance gap (success rate: 93.0% vs. 67.3%/66.3%; QNU score: 52.7% vs. 39.3%/28.3%) between MolEvol and [FixM]/[FixR] justiﬁes the necessity of taking both E-step and M-step into consideration.

Original: Compared with [MCTS], the 5% QNU increase may result from the larger space when doing the local search, while MCTS only proposes connected subgraphs of molecules as rationales.
Revised: Compared with [MCTS], the 5% QNU increase may result from the larger space when doing the local search, while MCTS only proposes connected subgraphs of molecules as rationales.

Original: Distribution of the Generated Molecules.
Revised: Distribution of the Generated Molecules.

Original: In Figure 3-left we plot the evolution of the generative model performance over time.
Revised: In Figure 3-left we plot the evolution of the generative model performance over time.

Original: As we can see, the distribution gradually shifts to regions with higher property scores, which demonstrates that MolEvol does improve the molecule generative model via EM iteration.
Revised: As we can see, the distribution gradually shifts to regions with higher property scores, which demonstrates that MolEvol does improve the molecule generative model via EM iteration.

Original: As shown in Figure 3-right, MolEvol can propose molecules with improved
Revised: As shown in Figure 3-right, MolEvol can propose molecules with improved

Original: QED and SA compared to molecules in ChEMBL and the reference set.
Revised: QED and SA compared to molecules in ChEMBL and the reference set.

Original: The distribution for the property scores of molecules generated by MolEvol is more compact than others, which suggests that MolEvol can propose molecules with high property score and low score variance.
Revised: The distribution for the property scores of molecules generated by MolEvol is more compact than others, which suggests that MolEvol can propose molecules with high property score and low score variance.

Original: Example of Rationale/Generated Molecule.
Revised: Example of Rationale/Generated Molecule.

Original: Figure 4 gives an example of molecules generated by some rationale discovered using MolEvol .
Revised: Figure 4 gives an example of molecules generated by some rationale discovered using MolEvol .

Original: The molecules are of high diversity and pertain consistently high level of scores, which proves MolEvol ’s superiority. 
Revised: The molecules are of high diversity and pertain consistently high level of scores, which proves MolEvol ’s superiority. score=0. score=0.

Revised: Rationale 𝑠

Revised: Molecule 𝑔 (cid:2869)

Revised: Molecule 𝑔 (cid:2870) Molecule 𝑔 (cid:2871) 𝑔 ~ 𝑝 (cid:2968) (cid:4666) · | s (cid:4667)

Original: [Figure]
Revised: [Figure]

Original: Human Evaluation.
Revised: Expert Evaluation.

Original: We asked an experienced chemist to evaluate generated molecules.
Revised: We asked an experienced chemist to evaluate generated molecules.

Original: The top-scoringmolecules from MolEvol and ChEMBL are selected, shufﬂed, and grouped with one another to constructpairs.
Revised: The top-scoringmolecules from MolEvol and ChEMBL are selected, shufﬂed, and grouped with one another to constructpairs.

Original: Given a pair of molecules, the chemist is asked to give a comparative score in each of the four criteria.
Revised: Given a pair of molecules, the chemist is asked to give a comparative score in each of the four criteria.

Original: For the sum of four scores, 30 /50 molecules by MolEvol are higher or identical compared to their counterparts from ChEMBL.
Revised: For the sum of four scores, 30 /50 molecules by MolEvol are higher or identical compared to their counterparts from ChEMBL.

Original: For individual scores, 7 /50 molecules by
Revised: For individual scores, 7 /50 molecules by

Original: MolEvol are all higher or identical compared to their counterparts.
Revised: MolEvol are all higher or identical compared to their counterparts.

Original: This shows that our algorithm can propose high-quality realistic molecules that are competitive with existing ones.
Revised: This shows that our algorithm can propose high-quality realistic molecules that are competitive with existing ones.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: In this paper, we proposed an EM-like algorithm for optimizing molecules by an explainable evolutionary process.
Revised: In this paper, we proposed an EM-like algorithm for optimizing molecules by an explainable evolutionary process.

Original: Although we focus our paper and evaluation on molecule design, the method can be generically applied for optimizing discrete structures in other structured prediction domains like program synthesis (Ellis et al., 2020) and graph adversarial learning (Dai et al., 2018a; Z¨ugner et al., 2018).
Revised: Although we focus our paper and evaluation on molecule design, the method can be generically applied for optimizing discrete structures in other structured prediction domains like program synthesis (Ellis et al., 2020) and graph adversarial learning (Dai et al., 2018a; Z ¨ ugner et al., 2018).

Original: Our method mimics a general design process of humans for discrete structures by ﬁrst identifying useful structural elements and then improving the design based on these elements.
Revised: Our method mimics a general design process of humans for discrete structures by ﬁrst identifying useful structural elements and then improving the design based on these elements.

Original: The process of discovering more useful substructures and then reiterating the design is carried on to gradually improve the ﬁnal product.
Revised: The process of discovering more useful substructures and then reiterating the design is carried on to gradually improve the ﬁnal product.

Original: Furthermore, the explainable graph model we developed in the paper can also be applied to other general graph problems as well.
Revised: Furthermore, the explainable graph model we developed in the paper can also be applied to other general graph problems as well.

Original: We believe multiple aspects of our method have broader applications beyond the current molecule optimization problem.
Revised: We believe multiple aspects of our method have broader applications beyond the current molecule optimization problem

Original:  A. D ETAILS OF E XPLAINER
Revised: A A PPENDIX A. D ETAILS OF E XPLAINER

Original: Here we clariﬁes the details for the explainer used in the “explainable local search”.
Revised: Here we clariﬁes the details for the explainer used in the “explainable local search”.

Original: For the MPN of h φ in the explainer, we use a two-layer GCN with embedding size of each layer equaling 32.
Revised: For the MPN of h φ in the explainer, we use a two-layer GCN with embedding size of each layer equaling 32.

Original: The input of the GCN is node embedding derived by an Embedding Layer that embeds each node (atoms) within the graph (molecule) into a 128-dim vector according to its category.
Revised: The input of the GCN is node embedding derived by an Embedding Layer that embeds each node (atoms) within the graph (molecule) into a 128-dim vector according to its category.

Original: The
Revised: The

Original: FC layer following MPN outputs a 1-dim vector which is used for Gumbel Softmax.
Revised: FC layer following MPN outputs a 1-dim vector which is used for Gumbel Softmax.

Original: For the MLP of q φ in the explainer, we use a two-layer FC network to embed the information, each of whose hidden dimension equals to 200.
Revised: For the MLP of q φ in the explainer, we use a two-layer FC network to embed the information, each of whose hidden dimension equals to 200.

Original: We add batchnorm after each FC layer to make the training phase more stable.
Revised: We add batchnorm after each FC layer to make the training phase more stable.

Original: After that, a sigmoid layer is used to get the ﬁnal prediction.
Revised: After that, a sigmoid layer is used to get the ﬁnal prediction.

Original: We give further details about the training procedure in Algorithm 3.
Revised: We give further details about the training procedure in Algorithm 3.

Original: Input: Molecules dataset D with each pair ( g, y ) denoting molecule and label, initial vertex sampling policy network φ , MLP network ψ for approximating Q . 1 for i ← 1 to N epochs do Sample g 1 , · · · , g m from D . [Equation] g
Revised: Algorithm 3:

Original: [Figure] 10 Round 4. O b j e c t i v e Va l u e MolEvol
Revised: V

Original: A. B ASELINES
Revised: A. B ASELINES

Original: We compare MolEvol against state-of-the-art molecule optimization algorithms below:
Revised: We compare MolEvol against state-of-the-art molecule optimization algorithms below:

Original: • RationaleRL (Jin et al., 2020) learns a graph completion model, but relies on a ﬁxed set of multiproperty rationales composed by single-property rationales extracted by MCTS.
Revised: • RationaleRL (Jin et al., 2020) learns a graph completion model, but relies on a ﬁxed set of multiproperty rationales composed by single-property rationales extracted by MCTS.

Original: Concretely, each state in MCTS represents a subgraph of the molecule and the reward function is deﬁned as the property score of the subgraph.
Revised: Concretely, each state in MCTS represents a subgraph of the molecule and the reward function is deﬁned as the property score of the subgraph.

Original: • REINVENT (Olivecrona et al., 2017) learns a RNN model with Reinforcement Learning for generating molecules in the form of SMILES strings; • MSO (Winter et al., 2019) optimizes the property using Particle Swarm Optimization (PSO) (Kennedy & Eberhart, 1995) in a continuous latent space of molecules.
Revised: • REINVENT (Olivecrona et al., 2017) learns a RNN model with Reinforcement Learning for generating molecules in the form of SMILES strings; • MSO (Winter et al., 2019) optimizes the property using Particle Swarm Optimization (PSO) (Kennedy & Eberhart, 1995) in a continuous latent space of molecules.

Original: • GA-D(t) (Nigam et al., 2020) employs a genetic algorithm enhanced with a neural network based discriminator component to promote diversity.
Revised: • GA-D(t) (Nigam et al., 2020) employs a genetic algorithm enhanced with a neural network based discriminator component to promote diversity.

Original: The discriminator used here tries to distinguish between molecules generated by the GA and the reference molecule set.
Revised: The discriminator used here tries to distinguish between molecules generated by the GA and the reference molecule set.

Original: The time-dependent adaptive penalty is also used for further promoting exploration.
Revised: The time-dependent adaptive penalty is also used for further promoting exploration.

Original: A. I MPLEMENTATION OF M OL E VOL
Revised: A. I MPLEMENTATION OF M OL E VOL

Original: We give the detailed implementation of MolEvol below for reproducibility.
Revised: We give the detailed implementation of MolEvol below for reproducibility.

Original: We ﬁrst pretrain the graph completion model p θ ( g | s ) on a dataset constructed from
Revised: We ﬁrst pretrain the graph completion model p θ ( g | s ) on a dataset constructed from

Original: ChEMBL (Gaulton et al., 2017), which contains over 1.4M drug-like molecules.
Revised: ChEMBL (Gaulton et al., 2017), which contains over 1.4M drug-like molecules.

Original: The pretraining dataset consists of 4.2M ( s, g ) tuples, where g is a random molecule from ChEMBL and s is a random subgraph of g .
Revised: The pretraining dataset consists of 4.2M ( s, g ) tuples, where g is a random molecule from ChEMBL and s is a random subgraph of g .

Original: In our experiment, MolEvol is run for 10 rounds.
Revised: In our experiment, MolEvol is run for 10 rounds.

Original: Within each round,rationales are added to the support set during the explainable local search stage.
Revised: Within each round,rationales are added to the support set during the explainable local search stage.

Revised: During the local search stage, 3 to 5 atoms will be sampled according to the vertex sampling policy

Revised: h φ ( g ) and we include the neighbors of the sampled atoms, i.e. the atoms which share a common bond to the sampled atoms, to form the rationale (Eq (13)).

Revised: In the molecule completion stage, the parameter θ is updated

Revised: [Figure]

Original: In the molecule completion stage, the parameter θ is updated with gradient descent for 1 epoch using a total number of 20000 ( s, g ) pairs with a minibatch size of 10 and a learning rate of 1e-3. 
Revised: 10 Round 4. O b j e c t i v e V a l u e MolEvol with gradient descent for 1 epoch using a total number of 20000 ( s, g ) pairs with a minibatch size of 10 and a learning rate of 1e-3. A. C ONVERGENCE A NALYSIS OF M OL E VOL

Revised: From a theoretical standpoint, here we assume 1) we use the true support set S instead of the ﬁnite support set S t in (Eq.

Original: A. M ORE E XPERIMENTS
Revised: 8), and 2) α and m in (Eq. 10) are carefully selected such that the gradient update has small enough variance. Proof • As J ( θ, p ( s )) has an upper bound, we only need to show that it is non-decreasing over E step and M step. • E-step: we need to show that J ( θ t , p t +1 ( s )) ≥ J ( θ t , p t ( s )) .

Revised: It is obvious with assumption 1) as (Eq. 6) has the closed form solution (Eq. 7), so the updated value of J is the maximum after the argmax operation.

Revised: • M-step: we need to show that J ( θ t , p t ( s )) ≥ J

Original: We conducted more experiments to see if MolEvol can generate more diverse molecules.
Revised: ( θ t − 1 , p t ( s )) . First, it is worth noticing we used the same trick as in REINFORCE to get Eq. 10 from Eq. 9, i.e. we can do SGD by the gradient we get in Eq.

Revised: Then, with assumption 2), by doing SGD, the unbiased gradient estimator with small variance will always converge to a non-decreasing result in the objective value.

Revised: • By the above analysis, we can justify that this EM-like method can converge to some local optimum.

Revised: Note that both assumptions are rather mild, since for assumption 1), S t grows with time t and gradually converges to S , and for assumption 2), a large enough m and a small enough α should sufﬁce.

Revised: As will be discussed later, we have a plot (Fig. 5) of the ﬁnal objective’s convergence curve to justify that empirically our algorithm can converge.

Revised: A. M ORE E XPERIMENT R ESULTS

Original: Molecule Distribution.
Revised: Molecule Distribution.

Original: We projected the generated molecules onto a two-dimensional space by tSNE (Maaten & Hinton, 2008) together with the reference molecules in Figure 5-left.
Revised: We projected the generated molecules onto a two-dimensional space by tSNE (Maaten & Hinton, 2008) together with the reference molecules in Figure 5-left.

Original: The molecules generated by MolEvol occupy the chemical space expanded by the reference molecules and its neighborhood regions.
Revised: The molecules generated by MolEvol occupy the chemical space expanded by the reference molecules and its neighborhood regions.

Original: Optimization Objective.
Revised: Optimization Objective.

Original: We plot the value of J ( θ ) in Equation 5 during training.
Revised: We plot the value of J ( θ ) in Eq (5) during training.

Original: As can be seen in Figure 5-right, the value of objective J ( θ ) is consistently improved, which shows that MolEvol does help to optimize the diversity promoting objective in the alternating fashion. 
Revised: As can be seen in Figure 5-right, the value of objective J ( θ ) is consistently improved, which shows that MolEvol does help to optimize the diversity promoting objective in the alternating fashion. Analysis of Baselines.

Revised: The main reason for MSO’s low performance is that it produced molecules with relatively low diversity, so most queries were wasted for evaluating highly similar molecules.

Revised: Therefore MSO is not well suitable for producing high scoring molecules with high diversity since there’s no regularization for the diversity of molecules it generates.

Revised: GA-D(t) involves the discrimination score to promote the unseen molecule’s generation.

Original:  A. D ETAILS OF H UMAN E VALUATION 
Revised: However, there is no guarantee that the generated molecules are dissimilar enough to be deemed as novel, thus leading the degradation of overall performance. In comparison with them, REINVENT and RationaleRL resort to REINFORCE for optimization, and achieves more competitive performance. Nevertheless, RationaleRL generates molecules from rationales in one-shot, which does not take the insight that the generated molecules might contains some subgraphs ( i.e. rationales) which are more qualiﬁed into consideration. A. E XPERT E VALUATION D ETAILS

Original: Here, we give more detailed setting for our human evaluation experiment:
Revised: Here, we give more detailed setting for our expert evaluation experiment:

Original: We 
Revised: We choose the top-50 molecules of the same

Original: choose
Revised: size from

Original: the top-50 molecules of the same size from our generative model and ChEMBL dataset.
Revised: our generative model and ChEMBL dataset.

Original: Then, We randomly shufﬂe the order for each group of molecules for pair comparison.
Revised: Then, We randomly shufﬂe the order for each group of molecules for pair comparison.

Original: Finally, we get 50 pairs of molecules and give these pairs to experts for evaluating.
Revised: Finally, we get 50 pairs of molecules and give these pairs to experts for evaluating.

Original: The experts evaluate each pair considering each of the four properties, i.e. GSK-3 β , JNK-3, QED, SA.
Revised: The experts evaluate each pair considering each of the four properties, i.e. GSK-3 β , JNK-3, QED, SA.

Original: For each property, the expert will give a judgment in three different choices: A. the former is clearly better; B. the latter is clearly better; C. the difference is minor, hard to tell.
Revised: For each property, the expert will give a judgment in three different choices: A. the former is clearly better; B. the latter is clearly better; C. the difference is minor, hard to tell.

Original: We report the following two metrics:
Revised: We report the following two metrics:

Original: • [M-Single]: The score of a molecule given single property is given by the following rules: a. if it is clearly better, get 2 scores; b. if it is hard to tell, get 1 score.
Revised: • [M-Single]: The score of a molecule given single property is given by the following rules: a. if it is clearly better, get 2 scores; b. if it is hard to tell, get 1 score.

Original: The ﬁnal score of the molecule is the score summed over four single property scores.
Revised: The ﬁnal score of the molecule is the score summed over four single property scores.

Original: We count the number of the molecules where the score of the ChEMBL molecule in this pair is not better than our generated one.
Revised: We count the number of the molecules where the score of the ChEMBL molecule in this pair is not better than our generated one.

Original: 30 out ofmolecules have scores that are not lower than the counterpart in this pair • 
Revised: 30 out ofmolecules have scores that are not lower than the counterpart in this pair • [

Original: [M-Overall]: We count the number of the molecules where each of four properties for the ChEMBL molecule in this pair is not better than our generated one.
Revised: M-Overall]: We count the number of the molecules where each of four properties for the ChEMBL molecule in this pair is not better than our generated one.

Original: We discard the pairs if there is no conﬁdent assertion, i e. four judgment is C for the pair.
Revised: We discard the pairs if there is no conﬁdent assertion, i.e. four judgment is C for the pair.

Original: 7 out of 50 molecules have no vulnerable property compared with the counterpart in this pair.
Revised: 7 out of 50 molecules have no vulnerable property compared with the counterpart in this pair.