Paper:
Challenges in natural sciences can often be phrased as optimization problems. Machine learning techniques have recently been applied to solve such problems. One example in chemistry is the design of tailor-made organic materials and molecules, which requires efficient methods to explore the chemical space. We present a genetic algorithm (GA) that is enhanced with a neural network (DNN) based discriminator model to improve the diversity of generated molecules and at the same time steer the GA. We show that our algorithm outperforms other generative models in optimization tasks. We furthermore present a way to increase interpretability of genetic algorithms, which helped us to derive design principles.

Review:
The paper describes a model for the generation and optimization of molecules (de novo molecular design), which combines a genetic algorithm based on the SELFIES string representation and a trained discriminator. 
The performance of the method is then shown on a small number of optimization tasks, and analyzed in detail.
Overall, this reviewer thinks this is a very interesting approach with a lot of potential, however, the experimental validation could be stronger.
Embedding in previous work:
Several key references should be added. In particular, this reviewer would suggest that the paper from the AZ group (https://arxiv.org/abs/1701.01329) is added, 
 which first introduced the language model (LM)-based class of molecule generation models, and the RL paradigm of a molecule-generating agent, which takes actions in an environment to construct a molecule, receiving reward from an external scoring function.
Also, the previous paper by Liu et al (“Constrained Graph Variational Autoencoders for
Molecule Design”, NeurIPS 2018,  https://arxiv.org/pdf/1805.09076.pdf ) should be cited, in particular, because it highlights the competitiveness of the LM based baselines.
In Table 2, it would be great to include a reference to the DEFactor paper from Yoshua Bengio’s lab (https://arxiv.org/abs/1811.09766), which achieves comparable performance to the strongest baseline.
In addition, the authors should cite earlier work on using GAs for de novo design, which have a rich tradition in chemoinformatics since the 1990ies.
Experimental:
Re 4.1. In the unconstrained optimization task, several baselines are missing. In particular, a language model + RL baseline should be considered. This model has also shown strong performance (in terms of optimization and quality) in the state of the art benchmark set (Guacamol https://pubs.acs.org/doi/10.1021/acs.jcim.8b00839 ) -  the code is available on github
Can the authors explain why they believe the physchem only function ”log(P) - SA(m) - RingPenalty(m)” is a relevant scoring function for the tasks that are faced in real life drug and materials discovery? It would be great if the authors would consider using the Guacamol benchmarks which have been designed by domain experts to allow a more fine grained assessment on tasks relevant to practical drug and materials design, while still being tractable computationally. Also, the guacamol paper provides a benchmark to assess the quality of molecules, which is currently missing in the present work.
Re 4.2. And 4.3.
The long term experiment and the molecule class exploration analysis are great, and well presented. I like the analysis of how the different classes emerge. However, are you sure that the GA is not just exploited the deficiencies of the rdkit logP calculator (which itself is just a model) by creating the molecules with the long sulfur chains? Also, I am not sure that it is likely that one would be able to synthesize these molecules in the lab.
Re 4.4
The results on the constrained optimization are impressive, however, it is not really clear if the GA is not simply exploiting deficiencies in the scoring function again.
Overall, this reviewer believes the proposed model is very promising (in particular also because GA’s are cool), however, the experimental validation could be more rigorous, which can be addressed by considering the guacamol benchmarks. If this results can provided, this reviewer will raise their recommendation to 'accept' (regardless of outcome of the benchmark)
#####
After response:
Considering the authors' response, I adapt my recommendation to accept. I acknowledge the short time the authors have  to run additional experiments. nevertheless, I would suggest that the authors at least run more challenging benchmarks later and report the results e.g. as a blogpost or in a leaderboard.