Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: SUMMARY:

The authors propose a method that "explains" molecular properties based on molecular fragments and call it Molecular Evolution. The subgraph "explanations" then are used to explore larger swaths of chemical space.

PROS:

- As far as the reviewer notes, this approach is novel in the (now increasingly crowded) set of alternatives for molecular generative models.
- The authors have a model that compares favorably to the baselines
- The authors use a very relevant set of optimization parameters for the multiobjective task.
- The paper is well explained.


CONS:
- The reviewer believes that there is much more to explain why a molecule is better for a task than identifying a subgraph. This should be made clear in the manuscript as materials scientists want to know for example quantum properties of the fragment(s) and how they influence the given property to provide a valuable explanation.

review_title: Review of Molecule Optimization by Explainable Evolution

final_score: rating: 7: Good paper, accept
review_content: ### Initial Review

The paper proposes a new algorithm for de-novo molecular design, which uses a model to extract explanatory subgraphs from a set of support molecules which "explain" high scores wrt a scoring function, and a generative model which is conditioned on these subgraphs to produce full molecules. 
 
All in all, I think this is an interesting combination of several existing approaches in generative models for molecules (all referenced in the paper). The aspect of explainability is novel. The presentation of the paper is mostly clear. The approach is quite geared to molecule generation, but can potentially also inspire applications in other domains, which makes it interesting from a general ML perspective as well. 
 
I like the paper from the theoretical side, which alone warrants acceptance of the paper at ICLR in my opinion. 
 
However, I have a few minor concerns / comments: 
 
I am a bit on the fence with the validation method. Since almost every new paper in the field proposes a new validation approach, it has become pretty much impossible to assess what the state of the art of the field is (or if the concept of SOTA is even something meaningful), and this paper is no exception in this regard. But I assume the authors will disagree here. 
 
In practice, generating 20k molecules is a lot. Looking at the statistics of the top100 molecules would probably be sufficient. 
 
Also, I find it somewhat surprising that some of the baseline algorithms (in particular the Winter et al MSO model), which are less constrained than algorithm presented here, are not achieving higher scores, in particular when the algorithms can query the scoring function 5 M times. Maybe this is something the authors could comment on in the rebuttal. 
 
As an additional baseline, I would suggest to report the "best in dataset", I.e. run the scoring function on the seeds and all molecules used to train the generator, and pick the top molecules. 
 
 
Related work: 
 
I would suggest to additionally cite https://arxiv.org/abs/1701.01329 which was the first paper to apply neural models to molecule generation in drug discovery, and the first of such papers which has been prospectively validated in laboratory experiments by scientists not affiliated with the authors.

### Update 1 after discussion:
Score raised.
review_title: novel approach for molecular design using explainability

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose a two step procedure for generating molecular graphs that optimize some desirable properties. The method consists of a rationale extraction phase, where the subgraph "important" for the desired property is identified and a graph completion step. 

When reading the paper for the first time, I found it a bit hard to follow the approach. The paper might be easier to read when the individual model components are introduced directly with the E- and M-step. Currently, the graph completion model is introduced in Sec 3.1, then the E- and M-steps are described in Sec. 3.2, while the "explainer" is only mentioned and referred to Sec. 3.3. An easier to follow structure might be: 1) Rationale extraction 2) Graph completion 3) E/M iteration.
Beyond that, the terms "evolution" and "explanation" might be misleading here: The method is not an evolutionary algorithm, as the title might suggest. Even if subgraphs are extracted that lead to the generation of promising molecules in the graph completion step, this does not show that these subgraphs responsible. To warrant the term explaination, a  thorough analysis of the extracted rationales would be necessary, in particular since they might not even be connected graphs.

The evaluation of the method in Table 1 and Fig. 3 show that the method outperforms previous appraoches and is capable of shifting the generated distribution shifts to higher scores. The human evaluation is weak, since only a single expert was asked. A panel of experts with reported agreement among the panel would improve the paper. However, since the scores of the molecules are available, it does not become quite clear to me, how human experts benefit the performance evaluation under those same criteria.

Pros
-----
- Interesting approach to update the pool of rationales
- Outperforms previous approaches, Fig 3. show that desired properties improve over rounds
- Ablation study demonstrates improvement by proposed procedure

Cons
----
- Structure of the paper could be improved
- The paper states that in the explainer a "subgraph s of k vertices" is extracted, therefore I assume the size of the rationale is fixed. This would severely restrict the space of rationales.
- The notion of explainability is not sufficiently discussed in the paper and the claim(?) that the rationales are somehow meaningful is not examined.

Update: I read the reply and thank the authors for the clarifications.
review_title: Nice idea, but structure & clarity could be better

final_score: rating: 7: Good paper, accept
review_content: The paper tackles the problem of molecule property optimisation. To this end, the authors proposes an alternating approach consisting of an explainer model and a molecule completion model. The explainer model takes a complete molecule as input and outputs a subgraph that represents the part that contributes most to property prediction. Then, the molecule completion model uses the subgraphs to sample a complete graph that can maximise the property scores. The loss function of molecule completion model directly maximises the properties, which is non-differentiable so that the authors use a REINFORCE algorithm for optimisation. 

Pros:

1. The paper proposes use subgraphs that contributes most to the property prediction for searching better molecules. The subgraphs are learned supervisedly through the signals from the property to be optimised. Compared to unconditional VAE models, this approach might be easier to optimise, since the subgraphs can serve as templates. 

2. The method is novel and the experiments demonstrate the effectiveness of the methods compared to previous methods.

3. The paper is well-written and the idea is articulated in a formal description.

Cons (or questions):

1. Some convergence analysis shall be needed, i.e. why this method will converge to the optimal values of the objective. The authors claim the method is an EM algorithm, and some proofs about convergence might be needed. Otherwise, some learning curves might be helpful, since the REINFORCE algorithm is known to suffer from high variances. 
review_title: The paper proposes an alternating algorithm to optimise molecule properties based on representative molecule subgraphs. The method is novel and achieves promising results.

Meta-review: The authors appreciated this submission because (a) the aspect of explainability is novel, (b) its strong performance, (c) the clarity of the paper. I urge the authors to double check all of the reviewer comments to make sure they are all addressed in the updated version. I vote to accept.
Decision: Accept (Poster)