Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary:
This work is about model evaluation for molecule generation and design. 19 benchmarks are proposed, small data sets are expanded to a large, standardized data set and it is explored how to apply new RL techniques effectively for molecular design.

on the positive side:
The paper is well written, quality and clarity of the work are good. The work provides a good overview about how to apply new reinforcement learning techniques for sequence generation. It is investigated how several RL strategies perform on a large, standardized data set. Different RL models like Hillclimb-MLE, PPO, GAN, A2C are investigated and discussed.  An implementation of 19 suggested benchmarks of relevance for de novo design will be provided as open source as an OpenAI Gym. 


on the negative side:
There is no new novel contribution on the methods side.  



minor comments:

Section 2.1. 
see Fig.2 —> see Fig.1
page 4just before equation 8: the the
review_title: This is a solid paper about model evaluation in the chemical domain. 

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes a set of benchmarks for molecular design, and compares different deep models against them. The main contributions of the paper are 19 molecular design benchmarks (with chembl-23 dataset), including two molecular design evaluation criterias and comparison of some deep models using these benchmarks. The paper does not seem to include any method development.

The paper suffers from a lack of focus. Several existing models are discussed to some length, while the benchmarks are introduced quite shortly. The dataset is not very clearly defined: it seems that there are 1.2 million training instance, does this apply for all benchmarks? The paper's title also does not seem to fit: this feels like a survey paper, which is not reflected in the title. Biologically lots of important atoms are excluded from the dataset, for instance natrium, calcium and kalium. I don't see any reason to exlude these. What does "biological activities on 11538 targets" mean? 

The paper discussed molecular generation and reinforcement learning, but it is somewhat unclear how it relates to the proposed dataset since a standard training/test setting is used. Are the test molecules somehow generated in a directed or undirected fashion? Shouldn't there also be experiments on comparing ways to generate suitable molecules, and how well they match the proposed criterion? There should be benchmarks for predicting molecular properties (standard regression), and for generating molecules with certain properties. Currently it's unclear which type of problems are solved here.

Table 1 lists 5 models, while fig 3 contains 7, why the discrepancy? In table 1 the plotted runs seem to differ a lot from average results (e.g. -0.43 to 0.15, or 0.32 to 0.83). Variances should be added, and preferably more than 3 initialisations used.

Overall this is an interesting paper, but does not have any methodological contribution, and there is also few insightful results about the compared methods, nor is there meaningful analysis of the problem domain of molecules either.

review_title: review

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: This paper studies a series of reinforcement learning (RL) techniques in combination with recurrent neural networks (RNNs) to model and synthesise molecules. The experiments seem extensive, using many recently proposed RL methods, and show that most sophisticated RL methods are less effective than the simple hill-climbing technique, with PPO is perhaps the only exception.  

Originality and significance: 

The conclusion from the experiments could be valuable to the broader sequence generation/synthesis field, showing that many current RL techniques can fail dramatically. 

The paper does not provide any theoretical contribution but nevertheless is a good application paper combining and comparing different techniques.

Clarity: The paper is generally well-written. However, I'm not an expert in molecule design, so might not have caught any trivial errors in the experimental set-up. 
review_title: empirical evaluation of recurrent models and RL for molecule design

Meta-review: The paper creates a dataset for exploration of RL for molecular design and I think this makes it a strong contribution to the community at the intersection of the two. For a methods focussed conference such as ICLR however, it may not be the best fit. Hence I would recommend submitting to a workshop track or targeting a more focussed venue such as a bioinformatics conference. 
Decision: Invite to Workshop Track