Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The authors present a method All SMILES VAE that’s used for predicting chemical properties of small molecules and also for optimizing the structures of these molecules. The authors evaluate their model on the Zinc250K and Tox21 dataset and report that they are able to exceed the previous SOTA. This paper was well written, and did a good job with explanations and illustrations. 

The central idea is to use a RNN to learn representations of strings encoding molecular structures (SMILE strings). SMILE strings are constructed by a DFS traversal over molecular graph structures. The authors choose to feed in distinct SMILE encodings of the same molecule in parallel, resulting in a stacked RNN architecture. They observe that since SMILE strings are DFS traversals of the molecular graph, propagation in the RNN corresponds to sequential message passing steps between nodes in the graph. This is in contrast to a graph neural network wherein all nodes simultaneously broadcast messages to their neighbors at every propagation step.

While it’s interesting to be able to optimize molecules in the space of SMILE strings, the impact is less clear. The authors mention in Sec 3.1, that graph models e.g. GCNs have higher overall compute complexity O(b^2), compared to their method which has O(Mb); however this does not do complete justice. Most graph propagation operations benefit from heavily parallelizable sparse matrix operations on GPUs. Infact, they can actually be much faster than RNNs in practice since GCNs need only a fixed number of propagation steps (independent of the number of bonds).

In Sec 3.2, the authors describe their approach for constraining the space of molecular optimization. While this may lead to directed searches, it will prevent truly novel molecules from being synthesized. The authors will need to address and provide experiments with unconstrained search.

It was not clear if the authors implemented any of the baselines? It seems from the text of figure 5, that the SSVAE and GraphConv results have been taken directly from the paper. The authors can strengthen their claim by replicating the results of baselines and making sure that they agree.

Some clarification questions:
 - For optimized/predicted strings, which are presumably novel molecules not in the dataset, how is the true chemical property  e.g. logP, determined?
 - In Figure 6, from steps 30 - 40, it seems that the predicted logP goes up but the true logP stays the same in a few cases. Why is this the case?

Overall, I think that this paper has some interesting ideas and is well written. However, the novelty and impact of the model is somewhat lacking. If this were introducing a new application area to this field, then I think the case for acceptance could have been stronger, however there has already been a lot of work related to molecular property prediction/design. 

Also, AC please note, I would have given the paper a score of 5, but Openreview only gave me a choice between 3 and 6. So, I went with 3, but please consider this to be a 5.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The authors describie a novel variational autoencoder like method for molecules. Instead of using graph neural networks, the authors hava an approach based on SMILES which encode molecules as strings. To avoid the problem that any given molecule may be represented by multiple SMILES strings, the authors consider an encoder that makes use of several random SMILES representations of the input molecule. These are preprocessed using recurrent neural networks generating an average representation by pooling the representations generated with each SMILES sequence for each atom in the input molecule. This reduces the number of operations needed to share information across all the atom in the
molecule (from N^2 in graph neural networks to MN in the proposed approach with M different SMILES representations of the input molecule). The model decodes then into a disjoint set of SMILES strings different from those used at the
input. This enforces the model to learn a bijective mapping between molecules and latent representations. The model trains also jointly a property regressor, linear or logistic. They do constrained optimization in the latent space, satying
within a reparameterized shell with most of the probability mass for the data. The proposed model can also do semi-supervised and supervised prediction tasks.

Clarity:

The paper is very clearly written and it is very easy to read. It contains a very detailed description of previous work.

Quality:

The proposed model is very reasonable and well-motivated. The experiments performed are exhaustive and informative enough to show that the proposed model and algorithms are useful in practice.

Novelty:

The proposed encoder/decoder model based on multiple SMILES representation is novel up to my knowledge.

Significance:

The experiments show that the proposed method can outperform previous ones. However, I miss additional evaluations using existing frameworks such as  Guacamol.

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper presents a VAE-based method to predict molecular properties as well as to design a novel molecular architecture.
The method employs as a basic input the SMILES representation of molecules which are not well defined in terms of the representation, though.
To tackle the issue, the authors formulate the method based on (1) multiple inputs of SMILES strings, (2) the character-wise feature fusion across those multiple strings and (3) network training by multiple output targets of SMILES strings different from the input ones.
As a result, the method provides the fixed-length latent representation which is robust against the variations of the SMILES representation and is useful for predicting the molecular properties and optimizing the molecular design.
The experimental results on the tasks of the property prediction and molecular optimization using the benchmark datasets demonstrate the effectiveness of the proposed method in comparison with the others.

Though this paper provides some contributions to the field of molecular graph representation, it contains flaws in presentation, lacking details of technical contents; thus, the paper is regarded as "borderline". I would like the authors to properly provide the details in the following points.

* This paper lacks some important technical contents, making it hard to understand. 
- What is the actual form of the property predictor used in p(\rho|z). And, in the first place of the paper, it would be better to explain what kind of and how many properties of the molecular are considered.
- How is the decoder constructed and trained? Though the authors state that a single-layer LSTM is employed as the decoder, there is no clear description how to cope with the multiple outputs (decoder targets) in training the LSTM.
- What does \theta in p_\theta(z) mean? In the VAE, p(z) works as a simple prior on z.
- The description, especially in Sec. 3 for the proposed method, is rather poorly presented by using less amount of math. At least, the authors should first depict the overall architecture of the method through clarifying the above-mentioned technical points.
- It is unclear how to apply the proposed method to the semi-supervised learning framework? Is it re-formulated in the SS-VAE [a] framework?
- The comparison experiments seem to be inconsistent. The proposed method is compared with different methods in different datasets/tasks. Toward fair comparison, it should be evaluated in comparison with some baselines including such as JT-VAE consistently.

[a] Kingma, D. P.; Mohamed, S.; Rezende, D. J.; Welling, M. Semi-Supervised Learning with Deep Generative Models. Proceedings of the 28th Annual Conference on Neural Information Processing Systems. 2014; pp 3581–3589.

* The authors cope with the fluctuations regarding the SMILES representation by means of RNN. There, however, are some approaches to canonicalize the SMILES representation itself such as by canonical SMILES. The authors have to mention those other approaches and discuss the superiority of the method over them; hopefully the proposed method should be compared with the naive one simply combining VAE [Kang&Cho18] and canonical SMILES representation. And to further understand the difficulty in SMILES variations, it would be better to show the averaged cardinality of SMILES representation per molecular.


Minor comment:
* In aggregating the RNN features for each character in SMILES strings, it might be possible to incorporate some structural knowledge into the process; for example, "C"s in benzene share the identical feature representation through fusing all those features.
review_title: Official Blind Review #1

Meta-review: The paper proposes All SMILES VAE which can capture the chemical properties of small molecules and also optimize the structures of these molecules. The model achieves significantly performance improvement over existing methods on the Zinc250K and Tox21 datasets. 

Overall it is a very solid paper - it addresses an important problem, provides detailed description of the proposed method and shows promising experiment results. The work could be a landmark piece, leading to major impacts in the field. However, given its potential,  the paper could benefit from major revisions of the draft. Below are some suggestions on improving the work:
1. The current version contains a lot of materials. It tries to strike the balance between machine learning methodology and details of the application domain. But the reality is that the lack of architecture details and some sloppy definitions of ML terms make it hard for readers to fully appreciate the methodology novelty. 

2. There is still room for improvement in experiments. As suggested in the review, more datasets should be used to evaluate the proposed model. Since it is hard to provide theoretic analysis of the proposed model,  extensive experiments should be provided. 

3. The complexity analysis is not fully convincing. Some fair comparison with the alternative approaches should be provided. 

In summary, it is a paper with big potentials. The current version is a step away from being ready for publication. We hope the reviews can help improve the paper for a strong publication in the future. 
Decision: Reject