From: ICLR.cc/2021/Conference/Program_Chairs
title: Final Decision
decision: Accept (Spotlight)
comment: This work proposes a method for generating candidate molecules using a novel fragment-based MCMC proposal mechanism.

Pros:
* Well-written paper
* Novel idea for an important application
* Very good empirical performance compared to the state-of-the-art in multi-objective molecule generation
* Careful ablation studies

Cons:
* Some details were missing (runtime, experimental details) and have been added to the revised version.

The authors engaged in an extensive discussion with the reviewers and modified their paper to address the reviewer concerns.

After discussions three reviewers recommend accepting the work and consider it a novel and useful contribution to the field.

One reviewer (Reviewer 3) is not satisfied by the authors comments and has concerns about the work regarding: asymptotic correctness of the sampling; fairness of the experimental comparison; and computational complexity.  The authors provide detailed justifications for their choices.  After looking at the discussion there are two factors:
1. technical arguments regarding the correctness of the sampling method; the authors justify the correctness by known results for adaptive MCMC methods, and the argument is sound, and the area chair fully accepts the authors' arguments as correct and applicable.
2. extend of the experimental evaluation and suitable baseline methods; this is partially subjective.  The authors provide extensive experiments in their work and justify exclusion of certain methods in that they do not easily apply to the multi-objective setting.  In addition, Reviewer 3 demands a comparison of generated molecules per time, which is plausibly useful, however, none of the prior works have used such a metric in a consistent manner and it is clearly challenging to do so fairly as such metric would depend on specifics of the implementation and computer.  The authors have updated their paper and added runtime information for their method.  The area chair fully accepts the authors' arguments and justification for the current experimental scope.

In summary the area chair considers the remaining concerns by Reviewer 3 as invalid; in particular, the authors have made extensive efforts to engage and educate the reviewer.

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to AnonReviewer3's further concerns (cont. 2) 
comment: > **Q: Complexity**: 1) In MARS, the sampling procedure can not be shared by different molecules. That means for each molecule, we need to sample independently, which costs lots of time and maybe too expensive; 2) When you calculate the running time you also need to take train time of MPNN into account instead of only sampling time.

A: Thanks for the reminder! We have updated the paper with measurements on running time. The running time of MARS, including both sampling and training MPNN, is 16 hours in total. MARS has the same magnitude of hours in running time as other methods. The wall-clock running time should be used with caution since it is not a reliable measurement of the computational complexity of any algorithm -- it depends on the implementation, the underlying library, the running hardware, and the operation system. Notice that it is more important to explore the chemical space in molecular generation – find novel and diverse bioactive molecules [1].

[1] Huggins et al., Rational Methods for the Selection of Diverse Screening Compounds, ACS Chem. Biol. 2011

> **Q: More Baselines**: 1) [Jensen et al, Chem. Sci. 2019] and [Ahn et al, Neurips 2020], it would be more convincing if the authors can compare MARS with some conventional heuristic search approaches under fair setting; 2) the single-objective optimization methods can also be used to evaluate multiple objective optimization tasks by sequentially optimizing a single property each time. I still think they need to be compared.

A: Thanks for the kind notes! We're not comparing with the NEWLY suggested heuristic search baselines (GB-GA & GB-GM-MCTS [Jensen, 2] and GEGL [Aha et al, 3]) because of the following two reasons: 

1. We have already included the most recently published GA-based work (i.e., GA+D [1]). GA+D shows better performance than GB-GA, as studied in [1]. GB-GA shows better performance than GB-GM-MCTS [2]. 
2. GEGL [3] will be officially published in December 2020, which is much after the deadline of ICLR. Similarly, MoFlow [4] officially appeared in KDD 2020 Aug. 23-27. Per ICLR 2021 Reviewer FAQ [5] (quoted below), they are considered contemporary work. 
Since our full paper deadline is Oct 2, if a paper was published on or after Aug 2, 2020, authors are not required to compare their own work to that paper. ''' 

About the single-objective optimization methods, we have already addressed this issue in our previous reply. Reviewer 3 mentioned, "a simple approach to leverage these methods for multiple property optimization is to sequentially optimize multiple properties, one property at a time." We agree that these are alternative approaches, but the hypothetical methods based on GraphAF and MoFlow are not verified in this setting in literature. Although we are interested in the suggested hypothetical approach, the validation of GraphAF and MoFlow in this sequential setting is yet another separate work. We want to emphasize that our goal is to jointly optimize multiple objectives for molecules. GraphAF and MoFlow with sequential optimization do not fall into this category.

[1] Nigam et al., Augmenting genetic algorithms with deep neural networks for exploring the chemical space, ICLR 2020

[2] Jensen, A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space, Chem. Sci. 2019

[3] Ahn et al., Guiding deep molecular optimization with genetic exploration, NeurIPS 2020.

[4] Zang and Wang, MoFlow: An Invertible Flow Model for Generating Molecular Graphs, KDD 2020.

[5] ICLR 2021 Reviewer FAQ https://iclr.cc/Conferences/2021/ReviewerGuide#faq

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to AnonReviewer3's further concerns (cont. 1)
comment: > **Q: Convergence and Ergodicity**: 1) If the convergence can not be guaranteed, the proposed method is simply an enumeration method -- all the performance gain is achieved due to more enumeration instead of exploring the chemical space efficiently; 2) the predictive MPNN will introduce bias in the transition kernel and will hurt convergence to the target distribution; 3) Local connectivity analysis can not guarantee the ergodicity and irreducibility of the Markov chain.

A: We thank the reviewer for additional comments.

1. **Convergence**: In this paper, we mainly focus on validating the hypothesis that MARS can find novel, diverse, and high-quality molecules. The experimental results support the claim of the effectiveness of MARS. **The theoretical convergence proof would be helpful but not necessary nor sufficient for the effectiveness of molecule generation**. Per your suggestion, we added an intuitive analysis of the convergence in the updated paper. Here we reiterate. MARS is a Metropolis-Hastings method with an annealing scheme and an adaptive proposal. The convergence of annealed MCMC is proved in [1]. MARS satisfies the condition by choosing a proper temperature cooling schedule. Roberts and Rosenthal [2], and Rosenthal [3] prove the ergodicity of adaptive MCMC under two conditions: containment and diminishing adaptation. MARS satisfies the containment condition as long as the maximum size of the molecules is bounded (which can be a large integer). MARS also satisfies the diminishing adaptation condition as long as we choose the proper learning rate decaying for the proposal learning (e.g., Adam, whose learning rate shrinks to zero eventually). Combining both, we can obtain the convergence of MARS to its target distribution. We would like to kindly remind Reviewer 3 that the full theoretical attributes of MARS are important and worth studying, but it is too long to include here. It is suitable for a separate paper. 
2. **Bias in predictive MPNN**: The term "bias" is still not clearly explained by AnonReviewer3. If the term "bias" refers to the gap between the target distribution and the proposal distribution, $q(x\mid \cdot)=\pi(x)$ is actually not a necessary condition in applying MCMC algorithms [1], no matter whether the proposal is implemented based on human-design rules or parameterized as models (i.e., $q_{\theta}(x\mid \cdot)$ as in MARS). In [1], it is stated that "In general, it is possible to use suboptimal inference and learning algorithms to generate data-driven proposal distributions." Such learned proposals are frequently employed in practical applications [4,5]. The different learned MCMC proposals will affect the mixing rate, i.e., how fast it converges, but not whether it will converge. 
3. **"The proposed method is simply an enumeration method."** MARS is inherently an MCMC method. There is a distinct difference between MCMC and enumeration. The MCMC (and MARS) never tries to extensively explore the full candidate space; rather, it tries to sample only a very small portion of candidates with high likelihood. In contrast, enumeration requires the evaluation of exponentially many candidates in discrete space. MARS achieves good performance because it explores the space more effectively. From the visualization of generated molecules (Figure 3), we can see that different methods will have different generating patterns. And in both RationaleRL and GA+D, the generated molecules tend to cluster together, while the positive molecules sampled by MARS is distributed more evenly in the chemical space. If MARS is simply an enumeration method, as you mentioned, MARS should not outperform GA in the experiments.

[1] Andrieu et al., An Introduction to MCMC for Machine Learning, Machine Learning 2003.

[2] Roberts, G. O. and Rosenthal, J. S. Coupling and ergodicity of adaptive MCMC. Journal of Applied Probability 44:458–475. 2007.

[3] Rosenthal, J. S. Optimal proposal distributions and adaptive MCMC. InHandbook of Markov Chain Monte Carlo (S. Brooks, A. Gelman, G. L. Jonesand X.-L. Meng, eds.). Chapman & Hall/CRC, London. 2011.

[4] Baele et al., Adaptive MCMC in Bayesian phylogenetics: an application to analyzing partitioned data in BEAST, Bioinformatics 2017.

[5] Tu and Zhu, Image segmentation by data-driven Markov chain Monte Carlo, IEEE Transactions on PAMI 2002.

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to AnonReviewer3's further concerns
comment: Thanks for the quick comments!

> **Q: Fairness of Comparison**: 1) it should be shown that MARS is fairly compared with baselines (i.e., with equivalent runtime), or the long running time of MARS is not an issue; 2) let all methods generate similar amounts of candidates, and allow all methods to query the oracle the same number of times. 

A: AnonReviewer3 suggested two setups for fair comparisons. However, these two strict requirements won't make so much sense in comparing methods that focus on practical drug discovery problems. This is because:

1. All methods should have **equivalent runtime**:
    1. As we mentioned in our replies previously, **the running time of molecular generation models (at most several hours) is ignorable comparing to the whole procedure of drug discovery process (usually takes months to years)**. And in molecular generation, it is more important to well explore the chemical space – find novel and diverse bioactive molecules [1].
    2. In most previous work [2,3,4,5], the comparison of running time is also not reported. Even the paper [6] mentioned by AnonReviewer 3 does not compare with this setup.
    3. Wall-clock running time can only be a rough reference but not a reliable metric since it highly depends on the algorithm implementation, the underlying library, the hardware, and the operating system. The hardware-specific implementation and their precise running time are interesting to study but outside the scope of this paper. 
2. All methods should **generate an equivalent amount of candidates and query the oracle the same number of times**: Thanks for the kind suggestion! 
    1. Notice that it is infeasible to count the exact times of querying the scoring functions (or oracle as referred to by Reviewer 3) as each method uses the scoring functions differently. Most methods cited in the paper evaluates the scoring function many times during the training stage, and it is repeated until certain stopping criteria satisfy. While MARS combines both training and inference together (this is a characteristic of Genetic-based algorithms and MCMC sampling).
    2. **All methods are equivalently evaluated based on $N=5000$ generated molecules in our experiments, so this is a fair comparison**. Though GA+D and MARS will produce molecules during heuristic search and sampling, those "intermediate" molecules are not included in method evaluation, which is standard practice for algorithms of this kind. 

[1] Huggins et al., Rational Methods for the Selection of Diverse Screening Compounds, ACS Chem. Biol. 2011

[2] Jin et al., Multi-Objective Molecule Generation using Interpretable Substructures, ICML 2020

[3] Nigam et al., Augmenting genetic algorithms with deep neural networks for exploring the chemical space, ICLR 2020

[4] You et al., Graph convolutional policy network for goal-directed molecular graph generation, NeurIPS 2018

[5] Jin et al., Junction tree variational autoencoder for molecular graph generation, ICML 2018

[6] Jensen, A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space, Chem. Sci. 2019

> **Q: About the training of MPNN**: (1) Is MPNN pre-trained or trained during sampling? What is the data for training MPNN? (2) What is the input and target of MPNN? There is a gap between your MPNN and p_add/p_delete; (3) In your answer, is it in a self-supervised setting?

A: 

1. The adaptive proposal (parameterized with MPNN) is trained during the sampling (i.e. on the fly), and the training data (i.e., training signals) is also collected during sampling according to the description in Section 3.2 (Adaptive training) and Algorithm 1. 
2. The input and target have also already been clarified in our previous replies and the submitted revision. As stated in Section 3.2 (Parameterizing with MPNNs), the input is the molecular graph of a given molecule $x$ (the molecule that is waiting to be transformed by the proposal), and the output are probabilities $p_\text{add}, p_\text{frag}, p_\text{del}$ computed with Equation 7-9. 
3. As AnonReviewer3 suggested, MARS trains the adaptive proposal in a self-supervised setting instead of merely enumerating molecules from the enormous chemical space. 

> **Q: About the fragment vocabulary details**

A: Thanks for the kind notes! Due to the page limitation, it is impractical to describe the constructed vocabulary in our paper thoroughly. So we have uploaded our extracted fragment vocabulary in the supplementary material. Please refer to that for more details.

From: ICLR.cc/2021/Conference/Paper3014/AnonReviewer3
title: Thanks for the quick response. Critical issues still remain unaddressed and are elaborated here.
comment: Thanks for your response. Most of the critical issues still remain unaddressed. They are elaborated below. 


> Fairness of Comparison

What you suggested could only demonstrate that machine learning based methods are much more efficient than the conventional process. However, it DOES NOT show MARS is fairly compared with baselines (i.e., with equivalent runtime), or the long running time of MARS is not an issue. In fact, any latent space based method can generate many molecule candidates by controlling the latent variables. By querying the oracles many times you could always improve the accuracy performance for any latent space based method.

For fair comparison, you need to fix the running time for all methods (e.g., 16 hours), let all methods generate similar amounts of candidates, and allow all methods to query the oracle the same number of times. Also, when you calculate the running time you also need to take train time of MPNN into account instead of only sampling time.

In addition to that, [Ahn et al, Neurips 2020] proposes a similar idea with MARS. Their basic modification operations contain atom_deletion, atom_addition, atom_insertion, atom_type_change, bond_order_change, ring_bond_deletion, and ring_bond_addition, more complete than MARS. What is your advantage over their paper? 

I think the authors need to compare MARS with either [Jensen et al Chem. Sci. 2019] or [Ahn et al, Neurips 2020] empirically. In my understanding, [Jensen et al Chem. Sci. 2019] leverages Monte Carlo Tree Search, [Ahn et al, Neurips 2020] uses reinforcement learning, while MARS does not use any supervision signal except evaluating acceptance probability. Thus, MARS looks like an enumeration-based method with some pruning techniques. It is validated by the empirical results that MARS costs longer time than other baselines. I wonder if there is a significant improvement of MARS compared with other heuristic search methods. 

----------------------------------------------------------------------------------------------------------------
> Details are Missing in Section 3.2

Some important information is still confusing: (1) Is MPNN pre-trained or trained during sampling? (2) What is the input and target of MPNN? In your answer, is it in a self-supervised setting? (3) What is the data for training MPNN? There is a gap between your MPNN and p_add/p_delete. 

Also, in experiments, the authors state that 1000 frequent fragments are used, more details are required, e.g., what are the features for these fragments that contain atoms, rings, and their mixture? 

----------------------------------------------------------------------------------------------------------------
> Convergence and Ergodicity

Thanks for the response. However, the convergence issue still remains since the predictive MPNN will introduce bias in the transition kernel and will hurt convergence to the target distribution. If the convergence can not be guaranteed, the proposed method is simply an enumeration method--- all the performance gain is achieved due to more enumeration instead of exploring the chemical space efficiently.  As mentioned, I also have ergodicity’s concern. Local connectivity analysis can not guarantee the ergodicity and irreducibility of the Markov chain. 

----------------------------------------------------------------------------------------------------------------
> Complexity

Complexity is also a big issue for MARS. In translation based or latent space based approaches, the model is trained once and then used for all molecules. However, in MARS, the sampling procedure can not be shared by different molecules. That means for each molecule, we need to sample independently, which costs lots of time and maybe too expensive.  
experiment setting and results 

----------------------------------------------------------------------------------------------------------------

>More Baselines

Thanks for fixing some issues. However, I think the authors need to add more related baselines. Specifically, as the authors mentioned, [Jensen et al, Chem. Sci. 2019] and [Ahn et al, Neurips 2020] are very close in ideas and need to be compared. In addition, it would be more convincing if the authors can compare MARS with some conventional heuristic search approaches under fair setting. 

In addition, the single-objective optimization methods can also be used to evaluate multiple objective optimization tasks by sequentially optimizing a single property each time. I still think they need to be compared.

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to the concerns on the critical issues
comment: Thanks to the quick comments! We will reply to these concerns respectively in the following paragraphs. Particularly, 

1. **Convergence**: The theoretical proof of convergence would be helpful but not sufficient for the effectiveness in finding the desired molecules. Still, we have also provided some informal analysis. 
2. **Comparison unfairness**: We argue it is not unfair because taking advantage of querying the oracle is one of the characteristics of sampling-based methods. And the ultimate goal of drug discovery is to generate molecules with high qualities regardless of different approaches.
3. **Running time**: We have provided the results in our revised submission;
4. **Baselines**: We did not compare MARS with the suggested baselines mainly because they are not methods that focus on multi-objective generation problems. 

------------

**Q: Convergence: 1) bias contained in the predictive model; 2) ergodicity analysis.**

A: Thanks for the kind note! As suggested, the theoretical analysis of convergence is important and interesting. However, in this paper, we mainly focus on proving that MARS can find novel and diverse molecules with high qualities, and we have demonstrated the effectiveness of MARS with substantial experimental results. The theoretical proof of convergence to the target distribution would be helpful but not sufficient for the effectiveness of finding the desired molecules. We provide an informal analysis below, yet a lengthy and rigorous theoretical analysis will be thoroughly examined in a separate paper:

1. Compared with standard MCMC algorithms, MARS still falls in the Metropolis-Hastings algorithm with an annealing scheme and an adaptive proposal, which results in inhomogeneous transition kernels. 
    1. The convergence of annealed MCMC is discussed in [1], and it could be guaranteed by elaborately choosing a temperature cooling schedule. 
    2. The convergence of inhomogeneity is discussed in the Handbook of MCMC, Chapter 4 [2]. According to the diminishing adaptation condition (Equation 4.6) in [2], we can ensure the convergence of inhomogeneous MCMC by making the difference of proposals in consecutive iterations diminish to zero. This is easily satisfied in MARS by using an optimizer (e.g., Adam) whose learning rate will shrink to zero eventually.
    3. By combining these two results, it yields the convergence of inhomogeneous annealed MCMC. The comprehensive theoretical analysis would be lengthy and not the major focus of this application paper. Therefore we will describe it in a separate paper. 
2. For the ergodicity of MARS, the connectivity of transition kernels is strong enough to support most drug discovery applications (i.e., MARS is capable of well-exploring the whole chemical space) because:
    1. We have employed a massive molecular database (i.e., ChEMBL) to construct our fragment vocabulary, guaranteeing a broad coverage of the essential elements in chemistry (e.g., -H, -C, -F, -O, =O, and aromatic rings).
    2. Our approach adopts a sequence of flexible molecular editing actions (adding and deleting) which can gradually generate complex fragments using the relative simple fragments in our vocabulary as building blocks and finally produce a complex drug molecule.
    3. Thus, MARS can render a nice coverage in the chemical space. Besides, users can also increase the connectivity by defining their own molecular databases and fragment vocabulary construction strategies.
3. For the bias in the predictive model, if the term "bias" refers to the gap between the target distribution $\pi(x)$ and the proposal distribution $q_{\theta}(x\mid \cdot)$, we agree that in MARS, the adaptive proposal is just an approximation to the target distribution. However, $q_{\theta}(x\mid \cdot)=\pi(x)$ is not a necessary condition in applying MCMC algorithms [1]. 

In addition, many research works that employ annealed or adaptive MCMC to practical applications have been published without formally proving their convergence. For example, [3] applies adaptive MCMC to Bayesian phylogenetics, and [4] applies data-driven MCMC to image segmentation. And no one has argued about the bias issue. Meanwhile, most previous works on molecular generation (e.g., RationaleRL) have also skipped the theoretical analysis in the ability of finding high-quality molecules. 

-------------

[1] Andrieu et al., An Introduction to MCMC for Machine Learning, Machine Learning 2003

[2] Rosenthal, J. S. (2011). Optimal proposal distributions and adaptive MCMC. InHandbook of Markov Chain Monte Carlo (S. Brooks, A. Gelman, G. L. Jonesand X.-L. Meng, eds.). Chapman & Hall/CRC, London.

[3] Baele et al., Adaptive MCMC in Bayesian phylogenetics: an application to analyzing partitioned data in BEAST, Bioinformatics 2017

[4] Tu and Zhu, Image segmentation by data-driven Markov chain Monte Carlo, IEEE Transactions on PAMI 2002

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to the concerns on the critical issues (cont.)
comment: **Q: Unfair Comparison: it is unfair to compare MARS with baselines in the manuscript since MARS calls oracle many more times.**

A: Thanks for your kind notes. We disagree that our comparison is unfair. MARS indeed calls oracle many times, but this is one of our advantages. It is not very meaningful to require the compared methods to have the same characteristics in all aspects. For example, RationaleRL takes extra advantage of pre-training from big data and domain knowledge, while MARS does not.

Besides, similar to MARS, genetic algorithm-based approaches (e.g., GA+D, [1, 2, 3, 4]) will also call the oracle during each heuristic search step. They are broadly applied in traditional drug discovery while no one has argued about the fairness issue. Compared with GAs, MARS utilizes the oracle more efficiently by adaptively learning the desired distribution during sampling and offering an effective mechanism to accept/reject proposal candidates. Therefore, the comparison in our paper is fair. 

[1] Nathan et al., A Graph-Based Genetic Algorithm and Its Application to the Multiobjective Evolution of Median Molecules, J. Chem. Inf. Comput. Sci. 2004

[2] Devi, Evolutionary algorithms for de novo drug design – A survey, Applied Soft Computing 2015

[3] Jensen, A graph-based genetic algorithm and generative model/Monte Carlo tree search for the exploration of chemical space, Chem. Sci. 2019

[4] Ahn et al., Guiding Deep Molecular Optimization with Genetic Exploration, arXiv 2020

**Q: No Running Time Information: the authors also did not report the running time for the proposed method v.s. baselines in their revision.**

A: We have reported MARS's running time and other methods in our revised paper (Section 4.2).

> In the GSK3β+JNK3+QED+SA setting, MARS takes roughly $T=500$ sampling steps and 16 hours in total to converge. The sampling curves are shown in Figure 5. 

> For other baselines, RationaleRL takes 5.7 hours to fine-tune the model, and GA+D takes 278 steps and 2.2h to achieve its best performance. However, one thing should be noted, both RationaleRL and GA+D are greatly accelerated by adopting parallel computation, which is not implemented in MARS yet. We believe MARS will have comparable computation efficiency if the conversions of molecules to graphs and molecular evaluations are computed parallelly. 

> In addition, comparing to the conventional drug discovery process, which usually takes months to years, the time we spent on molecular generation models is ignorable (at most several hours). And in molecular generation, it is more important to well explore the chemical space -- find novel and diverse bio-active molecules (Hugginset al., 2011). 

**Q: Missing State-of-the-art Baselines (graph2graph [ICLR 19], graphAF [ICLR 20], MoFlow [KDD 20]): The results from Jin et al 2020 are also directly copied from the original paper despite the experimental setting can be different.**

A: In this paper, we have compared MARS with several recent representative molecular generation methods (e.g., JT-VAE, GCPN, RaationaleRL, and GA+D), and we did not include the suggested baselines because:

1. Graph2Graph [ICLR 19] is Jin's (the author of RationaleRL [1]) previous work, which is not included as a baseline in RationaleRL [1]. Thus we do not include them for comparison in our multi-objective settings;
2. GraphAF [ICLR 20] and MoFlow [KDD 20] are both flow based graph represention and generation algorithms, which do not target for multiple objectives. Reviewer 3 mentioned "a simple approach to leverage these methods for multiple property optimization is to sequentially optimize multiple properties, one property at a time". We agree that these are alternative approaches but GraphAF and MoFlow are not verified in this setting. Although we are interested in the suggested hypothetical approach, the validation of GraphAF and MoFlow in this sequential setting is yet another separate work. We want to emphasize that our goal is to jointly optimize multiple objectives for molecules. GraphAF and MoFlow with sequential optimization do not fall into this category. We'd like to thank Reviewer 3 for pointing this out. We have added discussion in our revised discussion. 

In addition, we want to clarify that, it is a misunderstanding that our experiment settings are different from [1]: 1) we both optimize GSK3β, JNK3, and GSK3β+JNK3; 2) we are both interested in the success rate, novelty, and diversity metrics. So we directedly report some of the baseline results from [1]. While in Table 2, where the experiment settings are different (X+QED+SA), all the results are obtained through experiments by implementing all the baselines with their officially released code. 

[1] Jin et al., Multi-Objective Molecule Generation using Interpretable Substructures, ICML 2020

From: ICLR.cc/2021/Conference/Paper3014/AnonReviewer3
title: Thank you for your response. The Most critical issues remain unaddressed.
comment: Thank you for the efforts in addressing some of my concerns. However, the most critical issues remain unaddressed.


--- Convergence
First, the theoretical analysis in this manuscript is borrowed from the convergence of standard MCMC method and simulated annealing. However they can not be directly applied to the proposed MARS. This is due to the transition probability relies on p_add, p_delete, and p_frag, which are estimated via predictive models and thus contains bias. The analysis did not consider the bias. 

--- Ergodicity Analysis
Second,  the ergodicity analysis should be added since it is crucial to guarantee that the Markov chain can traverse the whole state space. 

--- Unfair Comparison, No Running Time Information
Third,  the concern about the fairness in comparison has not been addressed. The authors also did not report the running time for the proposed method v.s. baselines in their revision.

It seems in the experiment, the proposed MARS generates many more molecule candidates, and for each candidate, oracles are called to evaluate its desired molecular distribution in Eq(1). In feedback, O(NT) is about 2.5M. Considering the K properties, e.g., K=4 in GSK3β + JNK3 + QED + SA, the proposed MARS calls oracle 10 million times, which is very costly and will be computationally prohibitive in practice. Also it is also unfair to compare MARS with baselines in the manuscript since MARS calls oracle many more times. 


--- Missing State-of-the-art Baselines
Last, regarding the experiments and missing baselines, the authors argue that some single property optimization methods can not be used in multiple property scenario. This is not true.  A simple approach to leverage these methods for multiple property optimization is to sequentially optimize multiple properties, one property at a time.  
The authors also miss many state-of-the-art methods, like graph2graph [ICLR 19], graphAF [ICLR 20], MoFlow [KDD 20]. The results from Jin et al 2020 are also directly copied from the original paper despite the experimental setting can be different. 
To summarize, without proper and extensive experimental comparison with state-of-the-art methods, it is unclear whether the proposed method works better than other models. Thus from experimental perspective the manuscript is also not ready to be published.

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to your concerns on experiment setting and results
comment: **Q: Important baselines are missing. E.g., Graph2Graph and hierarchical generation**

A: Thank you for mentioning G2G and HierG2G, and we do recognize them as important works in the field of molecular generation. However, due to the different focuses of MARS comparing to the above-mentioned models, we did not include them in the baseline models, and more details are explained as follows.

  1. G2G and HierG2G are designed and proposed for single-objective molecular generation tasks, while we mainly focus on multi-objective molecular generation as it is a more practical scenario in drug discovery.
  2. G2G, HierG2G, and RationaleRL[3] are developed by the same research group, with RationaleRL being their most recent work focusing on multi-objective generation tasks. In our paper, RationaleRL is used as one of the baselines. As mentioned earlier, due to the different focuses, works focusing on multi-objective generation tasks, such as RationaleRL, do not usually include models for single-objective generation as baselines, so does MARS. 

**Q: One baseline RationaleRL outperform the proposed MARS in terms of success rate in many settings.**

A: RationaleRL does perform better than MARS in some settings, but in the most challenging ones where bioactivity, drug-likeness, and synthesizability are all considered (i.e., X+QED+SA), the success rate of RationaleRL decreases significantly and falls behind GA+D and MARS, which indicates the deficiencies of RationaleRL in certain multi-objective generation scenarios. 

In addition, though the success rate is an important evaluation metric, it is not sufficient in determining the quality of a molecular generation system. Other criteria, such as novelty and diversity, are also crucial in evaluating molecular generation systems [1]. Therefore, we come up with the summary product score to provide a comprehensive evaluation considering multiple objectives. It is possible that in specific tasks, the performances of particular metrics are more important than the others, but in our paper, a more general scenario is the main focus. To make this clearer to the readers, we have included a discussion in our revision (Section 4.1, Evaluation metrics). 

**Q: How to use t-SNE to visualize molecule distribution?**

A: Thanks for your reminder! We follow [4] and use the ECFP6 fingerprints of positive sampled/generated molecules to generate the figure. In our revised submission, we have added the description in Section 4.2 (the last paragraph).

-------

[1] Hu et al, Strategies For Pre-training Graph Neural Networks, ICLR 2020

[2] Gilmer et al., Neural Message Passing for Quantum Chemistry, ICML 2017

[3] Jin et al., Multi-Objective Molecule Generation using Interpretable Substructures, ICML 2020

[4] Li et al., Multi-objective de novo drug design with conditional graph generative model, J.Chem.Info 2018

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to your concerns on details of Section 3.2
comment: **Q: What is the input for MPNN?**

A: The input for MPNN is the molecular graph of molecule $x$ as stated in the third paragraph in Section 3.2, i.e., the atoms in molecule $x$ will be aggregated as nodes, and the bonds will serve as the input for the edge network in MPNN. We have clarified this in our revised paper. 

**Q: How to determine the supervision signal p_add, p_delete for M_\theta(x)?**

A: These supervision signals are determined in a maximum-likelihood estimation way, and the data are collected through the sampling process. Intuitively, we will want the model to reproduce its previous success in editing molecular graphs. More specifically, supposing the model suggests adding a fragment $f$ onto the atom $u$ in molecule $x$, and the obtained molecule $x'$ gets a higher score. Then the tuple $(x, u, f)$ is added into the dataset to train the model to maximize the probability of linking atom $u$ and fragment $f$ for the input $x$.

**Q: When add/deleted fragments, how many fragments are added/deleted in a single transition kernel?**

A: Only one fragment is added/deleted in a single transition kernel. This is because we want to keep the model as simple as possible, as adding/deleting multiple fragments can be decomposed into multiple steps of adding/deleting one fragment.

**Q: The MPNN description (Eq.5--9) is borrowed from [1], please cite the paper. Also, the number of training samples also looks very limited according to [1].**

A: We will cite this paper, but there are also some differences in our settings: [1] proposes several graph pre-training tasks that can benefit the downstream tasks; While in MARS, instead of pre-training the model to adapt to various tasks, we want to find high-quality molecules w.r.t. a specific set of objectives. 

  1. For the citation issue, those equations are conventional in GNN literature [2] and are not one of our main contributions. We did not cite [1] previously because we were not aware of the relevance between our multi-objective molecular generation framework and the pre-training methods proposed by [1].
  2. For the limited number of training samples issue, as [1] focuses on pre-training, they will need a large amount of data to train the model. By contrast, we focus on generating molecules to some specific objectives, and the amount of our training samples is at the same level compared to previous work. Furthermore, MARS can achieve relatively high performance with the training data it collects during sampling according to the experiment results, which indicates the amount of training data we used in MARS is sufficient.

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to your concerns on convergence, comparison fairness, and complexity issues
comment: Thanks so much for your comments and constructive suggestions! We will respond to your concerns on the issues of theoretical guarantees, the comparison fairness, the computation complexity, and some method or experiment details respectively in the following paragraphs. Hope these replies could resolve your concerns, and any further comments are welcome!

--------

**Q: Lack of theoretical guarantee about the training and convergence: 1) The paper trains the MPNN during sampling procedure, which seems very challenging; 2) need to show the sampled molecules follow target distribution; 3) It is non-trivial to guarantee the convergence to target distribution.**

A: Thanks very much for your kind reminder! We have provided some further discussions on the training and convergence issues in Section 3.2 (Discussion on training and convergence). 

In brief, the sampling process will converge if the adaptive proposal itself converges. And this can be satisfied by employing an adaptive optimizer (e.g., Adam optimizer) to update the model parameters. Also, the stationary distribution of annealed MCMC will converge to the set of global optima if we set the temperature cooling schedule appropriately. 

> **Discussion on training and convergence.** On the one hand, for a MCMC sampling process based on an adaptive proposal to converge, it is required that the proposal $q_{\theta}$ will converge as the sampling time step goes to infinity (Rosenthal, 2011).  We can easily satisfy this requirement by choosing an appropriate training strategy (e.g., use an adaptive optimizer like Adam (Paszke et al., 2017) to update model parameters). On the other hand, most convergence results for simulated annealing typically state that, if for a give a temperature $T_i$, the homogeneous Markov transition kernel mixes quickly enough, then convergence to the set of global maxima of $\pi(x)$ is ensured for a temperature sequence $T_i = (C \ln(i+ T_0))^{-1}$, where $C$ and $T_0$ are problem-dependent (Andrieu et al., 2003). Therefore, we can elaborately design the cooling strategy of annealing to make a better approximation to the global optima.

**Q: Unfair comparison between the proposed sampling model and non-sampling baselines.**

A: Thanks for raising your concerns, but we think the comparison is fair for the following reasons:
1. For the comparison fairness in the evaluation, all the baselines have generated 5000 molecules, and the 5000 generated/sampled molecules are estimated as a whole set. We are not comparing the single best molecule. 
2. For fairness in querying the oracles, though we will utilize the oracle during sampling, other baselines will also have access to the oracles (e.g., labels for training a gaussian process predictor). Baselines like JTVAE need to do optimization in the latent space. GCPN needs to query the oracles for the reward.

**Q: complexity issue: How many times do the authors need to query the oracles? Calling oracles is expensive.**

A: In the sampling process, the system will query the oracle for $NT$ times, where $N$ is the number of sampling paths ($N=5000$ for our experiment), and $T$ is the total number of sampling steps ($T=500$ roughly for the average overall score getting converged). 

However, querying oracles multiple times is not as expensive as it seems: 

1. We can use ML models or expert-designed rules to estimate the molecules in a short time. For example, in our experiment (GSK3b+JNK3+QED+SA), the time we spent on querying the oracles is within 2 hours, and this is acceptable comparing to the time spent in the conventional drug design and discovery process, which usually takes months to years.
2. We can reduce the time in querying the oracles by a parallel implementation, as the property scores will not intervene with each other. 

Moreover, other baseline methods also need to query oracle many times. For example: 

1. When training GCPN and RationaleRL with reinforcement learning, the rewards are computed with oracles.
2. In translation-based or latent space-based approaches, a large amount of high-quality data is needed, and collecting them requires to query the oracles more times and even take expensive experiments.

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to your concerns on the details of the method
comment: Thanks so much for your comments and constructive suggestions! We will respond to your concerns on the selection of fragment vocabulary, the convergence speed, and some method details respectively in the following paragraphs. Hope these replies could resolve your concerns, and any further comments are welcome!

---------

**Q: The proposed strategy seems constrained by the selection of the fragment dataset.**

A: Thanks for your kind notes. Yes, the fragment vocabulary is curial for our proposed method, and an insufficient vocabulary could lead to a constrained strategy (e.g., some molecules can not be obtained by adding/removing fragment actions). However, this issue is addressed in our approach using the following strategies:

1. We have employed a massive molecular database (i.e., ChEMBL) to construct our fragment vocabulary, guaranteeing a broad coverage of the essential elements in chemistry (e.g., -H, -C, -F, -O, =O, and aromatic rings).
2. Our approach adopts a sequence of flexible molecular editing actions (adding and deleting) which can generate complex fragments and eventually complex drug molecules using the relative simple fragments in our vocabulary as building blocks.

It is possible that a tailored vocabulary can help MARS to produce better performances on specific tasks (e.g., selecting fragments based on the prior knowledge of the drug target). However, in this paper, we mainly focus on designing a general and flexible molecular sampling framework. For specific applications, we also provide a means for the users to define their own molecular databases and vocabulary construction strategies.

**Q: How fast is the convergence of the proposed method? What was the number of train steps used to converge MARS?**

A: In our GSK3b+JNK3+QED+SA experiment where $N=5000$ molecule sampling paths are simultaneously considered, MARS takes roughly 500 sampling steps (i.e., $T=500$) and 16 hours in total to converge. We have posted some plots with time steps on the X-axis, as you suggested in Section 4.2. 

As for other baselines, RationaleRL takes 5.7 hours to fine-tune the model, and GA+D takes 278 steps and 2.2h to achieve its best performance. However, one thing should be noted, both RationaleRL and GA+D are greatly accelerated by adopting parallel computation, which is not implemented in MARS yet. We believe MARS will have comparable computation efficiency if the conversions of molecules to graphs and molecular evaluations are computed parallelly. 

In addition, comparing to the conventional drug discovery process, which usually takes months to years, the time we spent on molecular generation models is ignorable (at most several hours). And in molecular generation, it is more important to well explore the chemical space -- find novel and diverse active molecules [1]. 

**Q: What nodes are taken for aggregation in MPNNs?**

A: Thanks for pointing out this! All the atom nodes in the molecule x are taken for aggregation in MPNNs (therefore, the input is from the molecule x only), and we have clarified this in Section 3.2 (Parameterizing with MPNNs).

We do this mainly for computation efficiency consideration. More computational time is needed if both fragments and molecule $x$ are considered since the number of combinations is proportional to the fragment vocabulary size, which is usually large. 

**Q: It is not stated in the text what is the initial molecule.**

A: All sampling paths are started with the identical initial molecule "C-C", which is also adopted by previous graph generation methods for organic molecules (e.g. [2]). We have clarified this in our revision (Section 4.1, Implementation details).

------

[1] Huggins et al., Rational Methods for the Selection of Diverse Screening Compounds, ACS Chem. Biol. 2011

[2] You et al. Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. NeurIPS 2018.

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to your concerns on the evaluation and the benchmark
comment: Thanks so much for your comments and constructive suggestions! We will address your concerns regarding the evaluation metrics, the baselines, and the benchmark respectively in the following paragraphs. Hope these replies could resolve your concerns, and any further comments are welcome!

------

**Q: The summary product score (success rate x novelty x diversity) could be a bit misleading for the casual reader.**

A: Thanks for your kind reminder! These metrics (i.e., success rate, novelty, and diversity) indeed capture different performance characteristics of models and should be considered thoroughly according to specific applications.

Along with these metrics, we propose the so-called summary product score mainly to provide an overall comparison of different methods. Generally, for drug discovery, it is the best case if a molecular generation system can simultaneously achieve high scores on all three metrics. This summary product score is meaningful by intuitively presenting the percentage of generated molecules being simultaneously bio-active, novel, and diverse, which are essential criteria to be considered in building a suitable drug candidate library in early-stage drug discovery [1]. This issue is also discussed in our revised submission (Section 4.1 Evaluation Metrics) following your suggestions. 

**Q: For the success rate metric, what was the reasoning for the thresholds?**

A: We set these thresholds by following previous work [2]. Intuitively, the reasoning for these thresholds is that, we require the generated molecules to be bio-active (GSK3$\beta$/JNK3 > 0.5 [2,3]), drug-like (QED > 0.6) and easy to synthesize (SA > 0.67). 

**Q: How did you implement the weighting for the multiple objectives (GSK3B/JNK3, QED, SA) in GCPN and JT-VAE?**

A: We equally weight all the objectives in all the settings for both our method and other baselines. Moreover, we do this for two reasons: 

1. For comparison fairness, so that MARS and all the baselines will have a consistent goal;
2. We have also tried to tune the weights to improve GCPN and JT-VAE, but the results remained roughly the same. As for RationaleRL, we employed their implementation where the environment will give the agent a discrete reward only when all objective thresholds are satisfied. 

Based on the above-stated reasons, we reported the results obtained using equally weighted objectives in the paper. However, in different scenarios, we can practically tune these hyperparameters according to the various requirements. 

**Q: Do you have any data on the ability of the proposed model to optimize raw property scores? We may be interested getting the handful of best molecules.**

A: Thanks for your interest in the molecules generated by our model! We have provided some examples of generated molecules in our revised paper. Please refer to Appendix A for more details. 

**Q: Any thoughts about the applicability of some public molecule generation/optimization benchmarks for this work?**

A: Thanks for your kind suggestion on the GuacaMol benchmark, which could be a suitable platform to evaluate the proposed method as it includes multi-objective molecular generation tasks. However, we can hardly provide a comprehensive and reliable evaluation of all models on the GuacaMol benchmark in this paper, considering the tight rebuttal schedule. We are currently conducting the experiments and will release the results in our future publications. 

**Q: The abstract claim is a bit ambiguous, "[The method] outperforms the best prior methods by 100% in terms of success rate, novelty, and diversity of generated molecules."**

A: Thanks for your kind notes! We have rephrased this sentence as "in the most challenging setting where four objectives -- bio-activities to two different targets, drug-likeness and synthesizability -- are simultaneously considered, our method outperforms the state-of-the-art significantly in a comprehensive evaluation." in our revised version.

------

[1] Huggins et al., Rational Methods for the Selection of Diverse Screening Compounds, ACS Chem. Biol. 2011

[2] Jin et al., Multi-Objective Molecule Generation using Interpretable Substructures, ICML 2020

[3] Li et al., Multi-objective de novo drug design with conditional graph generative model, J.Chem.Info 2018

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Thanks for your positive comments!
comment: Thanks so much for your positive and insightful comments! We highly appreciate your efforts and time in evaluating our paper. In the future, we are interested in applying our proposed MARS in practical drug discovery scenarios and further improving it by developing more adaptive training strategies and more advanced parameterization approaches for the molecular editing proposal.

From: ICLR.cc/2021/Conference/Paper3014/Authors
title: Response to all the reviewers and area chairs
comment: We would like to express our sincere appreciation to the reviewers and the area chair for your efforts and time spent reviewing our paper and the constructive, critical, and meaningful suggestions for our method. In the previous paragraphs, we have addressed all the reviewers' comments accordingly, with a special focus on the suggestive comments from reviewer 3 covering the training and convergence details, the comparison fairness, and the computational complexity. In addition, revisions are made in our submission to clarify our approach further and discuss these thoughtful suggestions.

Specifically, we have made the following changes in our revised submission (highlighted in red in the paper):
1. We have provided some further discussions on the convergence and theoretical analysis in Section 3.2 (Discussion on training and convergence), as mentioned as one of reviewer 3's main concerns;
2. We have provided more details about the experiment results:
    1. We posted some sampling curves (step-metrics and step-SR) in Section 4.2 (Figure 5) to show the convergence of MARS;
    2. We provided some discussion on the running time in Section 4.2 (the last paragraph);
    3. We provided some property score distributions of sampled molecules in Appendix A;
    4. We provided some examples of sampled molecules in Appendix B;
    5. We provided the extracted fragment vocabulary in the supplementary material. 
3. We have revised some descriptions in the paper:
    1. We revised the description for our experiment results in the abstract and introduction to reduce ambiguity. 
    2. We clarified the input of MPNNs in Section 3.2 (Parameterizing with MPNNs);
    3. We added a citation to the paper "Strategies for Pre-training Graph Neural Networks" in Section 3.2 (Parameterizing with MPNNs);
    4. We explained why we are interested in the summary product score in Section 4.1 (Evaluation metrics);
    5. We clarified the initial molecules of the sampling process in Section 4.1 (Implementation details).

From: ICLR.cc/2021/Conference/Paper3014/AnonReviewer1
title: Review of MARS for multiobjective drug discovery optimization
review: SUMMARY: The authors present an elegant Markov-Chain Monte Carlo (MCMC) method to carry out the task of generating molecular structures that satisfy several objectives.

PROS: 
- The work is well written, concise and easy to follow
- The methodology is competitive with other approaches that are state-of-the-art in the optimization of single properties (such as GA-D) and show that they outperform them in most cases.
- The references that it cites are balanced.
- The multiobjective optimization is based on biological objectives

CONS:
- I see no major cons with this work.
rating: 8: Top 50% of accepted papers, clear accept
confidence: 5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: ICLR.cc/2021/Conference/Paper3014/AnonReviewer2
title: Recommendation to accept
review: ##########################################################################

Summary:

This paper proposes a method to generate molecular graphs with multiple optimized properties. Molecular graphs are constructed/edited by the iterative addition and removal of molecular fragments. A MCMC search procedure, guided by a learned graph neural network that proposes good graph edit actions, is used to sample molecules with optimized properties. The proposed model is compared with some baselines on a few multi-objective optimization tasks and shows good performance.

##########################################################################

Reasons for score: 

Overall, I vote for acceptance. The paper presents an interesting method for multi-objective molecule optimization that shows good performance in the evaluation. However, I have some concerns about the benchmark tasks and baselines that hopefully could be addressed

##########################################################################

Strengths:

*Paper is written in a clear way, and is well structured

*Proposed model shows very good performance in the evaluation, compared to other baselines

*Ablation studies that provide useful insight about the model

Weaknesses:

*Some concerns about the benchmark tasks and baselines (see below)

##########################################################################

Questions and other comments:

*I think that the summary product score (success rate x novelty x diversity) could be a bit misleading for the casual reader. Looking at Table 2, it is clear to me that different models have different performance characteristics in relation to the success rate, novelty and diversity model performance metrics. In a practical use case, one may not equally weight the importance of these performance metrics, and instead decide on a trade off depending on the specific problem. Eg based on Table 2, we would pick the MARS model for high success rate, GA+D for high novelty, RationaleRL for high diversity. However, the simple summary product score (especially with the highlighted column in Table 1 and 2) make this nuance harder to see.

*The abstract claim: "[The method] outperforms the best prior methods by 100% in terms of success rate, novelty, and diversity of generated molecules." is a bit ambiguous, since it seems to only happen with the product score in 1 of the 6 tasks

*Do you have any data on the ability of the proposed model to optimize raw property scores? Although it is useful to see the proportion of molecules generated containing properties above a predefined threshold, sometimes we may be interested getting the handful of best molecules. Eg the most potent inhibitor, or to a lesser extent the molecules with the highest QED and SA scores.

*For the success rate metric, what was the reasoning for the thresholds? Eg GSK3B/JNK3 > 0.5, QED > 0.6, SA > 0.67

*How did you implement the weighting for the multiple objectives (GSK3B/JNK3, QED, SA) in GCPN and JT-VAE? It seems that the weighting of the different objectives is another set of model hyperparameters that requires appropriate tuning.

*Any thoughts about the applicability of some public molecule generation/optimization benchmarks for this work, eg https://github.com/BenevolentAI/guacamol?  



rating: 6: Marginally above acceptance threshold
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct

From: ICLR.cc/2021/Conference/Paper3014/AnonReviewer4
title: Multi-objective molecules optimization using MCMC
review: Summary:

The authors propose a novel way to generate molecules with specified objectives, named MArkov moleculaR Sampling (MARS). The idea of MARS is based on generating the chemical candidates by iterative editing fragments of molecular graphs. To transform a molecule x into another molecule x′, the authors considers two sets of graph editing actions fragment adding and fragment deleting, where fragments are connected components in molecules separated by single bonds. To generate the molecules with desired objectives, MARS is using Markov chain Monte Carlo sampling with specified annealing scheme, together with graph convolutional neural network. The results reported in the following paper are very promising and show that this could be a good direction in the area of multi-objective molecules optimization.
 

=============================================================================

Pros:

1. The idea proposed by authors seems novel. It is the combination of MCMC based on molecules, together with message passing neural networks.

2. The proposed model is a new state of the art in the area of multi-objective molecules optimization. In the experimental section the authors shows that molecules generated by MARS have the highest desired objectives in 5 out of 6 proposed tasks. Simultaneously the generated set of molecules seems novel and diverse.

3. The molecules generated by MARS are evenly distributed in the space with a range of novel regions covered, as showed in Figure 3. This is a desirable behavior, better than in the other generative models, where we can see clusters of generated molecules.

=============================================================================

Cons:

1. The proposed strategy seems constrained by the selection of the fragment dataset.

2. The instruction of calculating probability densities over fragments in the vocabulary is not clear. More precisely, the authors states that hidden state for fragment graphs is given by h^{graph} = MaxPooling({h^{node}_{u}}), however they do not state whether the nodes taken for aggregation are from the fragment itself or from combined fragment and molecule x or whether from the molecule x only.  


=============================================================================

Questions during rebuttal period:

1. It is not stated in the text what is the initial molecule. Do you have some starting set of molecules or always start from the same?

2. How fast is the convergence of the proposed method? It would be nice to see some plot with time steps on X axis and score on Y axis.

3. The authors did not state what was the number of train steps used to converge MARS in their experiments. What number of time steps did you use? How long does the MARS training takes, compared to other methods?

=============================================================================

=============================================================================

Reasons for score: 
 
Overall, I vote for accepting this paper. The problem of multi-objective molecules optimization is hard and really important in cheminformatics. The idea proposed by the authors is novel and confirmed experimentally. Hopefully the authors can address my concern in the rebuttal period. 

rating: 7: Good paper, accept
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct

From: ICLR.cc/2021/Conference/Paper3014/AnonReviewer3
title: review
review: Summary: The paper proposes a sampling-based approach for multiple property optimization in molecule generation.

Strength: the sampling approach is an interesting new direction for molecule generation. Also multi-objective molecule generation is an important task.

Weakness: I have several concerns regarding this paper.

(1) Section 3.2 is unclear, with lots of details missing. For example,
--- what is the input for MPNN? i.e., x in M_\theta(x).
--- How to determine the supervision signal p_add, p_delete for M_\theta(x)?
--- When  add/deleted fragments, how many fragments are added/deleted in a single transition kernel?
--- If multiple fragments are added/deleted, are they added/deleted sequentially or following some particular rule?
--- The MPNN description (Eq.5--9) is borrowed from [1], please cite the paper.  
--- The paper trains the MPNN during sampling procedure, which seems very challenging, in the first several steps the model is from scratch, how to guarantee the sampled molecule is reasonable? If initially, the sampled molecules can not provide informative supervision, it will be hard to train a strong MPNN model. Also, the number of training samples also looks very limited. In [1], it requires a large amount of existing drug molecules to pretrain a good MPNN model. 

(2) Unfair comparison between the proposed sampling model and non-sampling baselines. For example, when sample size N=5000, the proposed model will query oracle to select a subset of generated molecules. 
However, baselines such as JTVAE directly output the generated molecules, all of which are kept. Therefore, in performance comparison if you compare the best molecules in 5000 molecules with a single molecule generated by JTVAE, this is unfair. Please clarify.

(3) complexity issue: How many times do the authors need to query the oracles when performing optimization using the proposed sampling algorithm? How does this compare to the baselines? Each time you sample a new molecule, you need to evaluate the acceptance rate in Eq.2, while such an evaluation requires evaluation of target distribution in Eq.1 as well as need to call the oracle (e.g., evaluating the property), which is very expensive.

(4) Lack of theoretical guarantee. The authors need to show the sampled molecules follow target distribution. The authors also argue they adaptively train the MPNN, so the transition kernel will change during sampling. It is non-trivial to guarantee the convergence to target distribution. 

(5) There are also issues with experimental setting and results. For example
--- Important baselines are missing. For example, Graph2Graph [2] and hierarchical generation [3].
--- One baseline RationaleRL outperform the proposed MARS in terms of success rate in many settings.
--- The fairness in model comparison issue I mentioned in (2)
--- How to use t-SNE to visualize molecule distribution? It needs more details.

[1] Hu et al, ICLR 2020, Strategies For Pre-training Graph Neural Networks
[2] Jin et al, ICLR 2019, Learning Multimodal Graph-to-Graph Translation for Molecule Optimization.
[3] Jin et al, ICML 2020, Hierarchical Generation of Molecular Graphs using Structural Motifs.
rating: 4: Ok but not good enough - rejection
confidence: 5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature