Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: **Summary**

This paper proposed a way to modify the molecule based on language pretraining techniques. The sequence representation of molecules is based on IUPAC names, which can be more semantically meaningful and much easier to model than the SMILES or graph based molecule representation. The pretraining is done via a conditional text generation model where the model predicts the fragment names based on the remainder of the molecule and corresponding property values. The application on downstream molecule property optimization tasks show that the proposed approach is effective at obtaining high quality molecules.

main_review: **Comments**

Overall the paper presents an interesting treatment for the molecule property optimization (e.g., lead optimization, etc). The approach is simple but seems to be effective. The IUPAC naming alternative to SMILES can be of its own contribution to a broader context of chemical engineering. 

However I have several concerns regarding the current draft.

1. The technique contribution is not very significant. The IUPAC naming is interesting but itself is not solid enough to be considered as technical innovation. The masked pretraining has been seen in other context like NLP, program language, etc. Nevertheless, I think it is ok for an application paper if the quality of experiments is high and the results are solid. So I would focus more on the experimental part. 

2. The only baseline studied is Hierarchical G2G (Jin et.al) in Figure 6. Given that there has been many works in the space of molecule optimization, it would be necessary to include more baseline results, especially those with quantitative comparisons. Although some of the baseline methods may not be able to complete all the tasks, it would still be necessary to compare in the situations where baseline methods apply. In this way we can appreciate how significant the results are, especially given that the method proposed in this paper is self-supervised. 

3. I would also like to learn more about the limitations of this method. For example, the IUPAC limits the possibility of generating new functional groups. There must be a trade-off between model capacity and the variance of results. 

4. The extrapolation is not for free. I’d like to learn more about the failure cases where the model is not able to generate the molecules with desired properties. 

5. (Optional) while the model presents an unsupervised way of learning, it would be interesting to see if it can be further fine-tuned with the paired molecule data for lead optimization. In this way, one can compare the results directly with G2G in their benchmarks.

summary_of_the_review: **Review summary**

This is an interesting application paper in the domain of molecule optimization with some minor technical contributions. The preliminary results are interesting, but it would be more solid with more quantitative comparison with existing methods on existing benchmarks.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper utilizes a transformer architecture based on T5 to generate tokenized IUPAC names of molecules. This allows sampling molecules that are local modifications of starting ones, or are local in IUPAC name but might be chemically large while respecting an initial scaffold.

main_review: 
Strenghts
The idea of using the IUPAC nomenclature is indeed innovative and has not received attention in the area. The use of local modification of tokenized IUPAC names is exciting and seems to work for scaffold derivatization better than other methods

The T5 model is a powerful addition to language-based addition to molecule generation

Weaknesses
At a higher level, this paper proposes a new generative approach but does not evaluate it upon the mutually agreed benchmarks in the field. Guacamol or Moses have lots of meaningful tasks and splits, including some scaffold-based ones. It does not seem appropriate to switch architectures and tasks and then just compare with the one baseline. If the authors believe those benchmarks are relevant, then it is more appropriate to make a case why explicitly (and probably use them anyway)

Furthermore, making local modifications to starting scaffolds has a long history in cheminformatics. Approaches like genetic algorithms, including the graph-based genetic algorithm, can do a great job at local optimization challenges like this https://pubs.rsc.org/en/content/articlelanding/2019/sc/c8sc05372c And since all the properties come from a surrogate model oracle, it does not matter how many train pairs or in-time calls need to be made. LogP (or logD, i think for all the molecules shown they are __extremely__ correlated) is an additive property that can be improved by just appending more carbons to an alkyl chain, so it is a poor performance metric. One could very easily make a genetic-algorithm version of the token substitution approach as a baseline.

I have a number of smaller comments below
- The advantages of using IUPAC names are not clear at all. 

"a small change in a molecule can lead to a large change in the SMILES string" This is the case too for IUPAC names - taking one carbon away and closing a ring in the suggested structure in Figure 2 completely changes the IUPAC name

"flattening the graph into a list of atoms artificially creates variable- and long-range dependencies between bonded atoms" It does in the IUPAC naming too, since there's a canonical atom ordering that is needed to refer to for writing the molecular name

"It is difficult to reason about common substructures, because the same structure can be
represented in many different ways depending on how the graph was flattened"

Same in IUPAC nomenclature - the ester functionality is described through a completely different naming in the example below


OC(C1=C(OC(C)=O)C=CC=C1)=O
2-acetoxybenzoic acid

O=C(O1)C2=C(OC1=O)C=CC=C2
4H-benzo[d][1,3]dioxine-2,4-dione


"graphs do a poor job encoding symmetry, long-range interactions between atoms that are many bonds apart but nearby in 3D space, and long-range interactions that arise from conjugated system" Again, the IUPAC naming convention has many of these flaws too. There's no 3D whatsoever in IUPAC either.

"locants (e.g. “1,” “2,” “N”), which indicate connectivity," These locants are also based on arbitrary ordering, just like SMILES

Again just like SMILES, while there's only one canonical choice of IUPAC nomenclature based on prioritization rules, it is entirely possible to write the IUPAC name for a molecule correctly but based on another prioritization rule. As a matter of fact this paper's approach to tokenization likely does that, depending on the nature of the transformations it perform on the starting name.

Section 3.1 addresses few of the issues raised about SMILES / graphs. (for instance, it does address the multiplier, which is indeed very powerful in theory, but does not seem to be used by the generator in practice. "pentatrytilbenzene" would break the bank in logP)

"ignoring the embedding for “nitroso”" I would make this part of the sentence, rather than a footnote

"We discard any generations where the sentinel tokens do not line up, and we further discard any molecules that cannot be parsed by ChemAxon’s calculators. We also discard instances where C5T5 regenerates the base molecule" I think I understand the logic about needing to seed with something, but discarding the same from the statistics seems antithetic to the nature of a novelty metric. What is the fraction of molecules being discarded at every filter?

"Although this is not a preferred IUPAC name, it is still unambiguous, and therefore valid and parseable" This is exactly the point i was making earlier. It seems the approach struggles with synonyms just like SMILES.

"C5T5 generates molecules that are more synthetically accessible and similar to base molecules and that have a wider range of logP increases than HierG2G." I am not sure I'm looking at the plots right. How was synthetic accessibility computed? Is 0 high or low accessibility? The scale in RDKIT has 1 for easy molecule and 10 for difficult, So C5T5 is producing more difficult molecules. Same for the logP increase, i see HierG2G producing a mean of 4 or so, and C5T5 is a little over 1? 



"Methods like HierG2G that train on pairs of similar molecules are fundamentally
limited by a paucity of experimental molecu pairs that have high similarity and high property
value improvement" This seems misleading since all these methods are using surrogate models to make their labels, there's essentially N squared pairs for the training library. 



Questions

"The nearest neighbor of “diphosphate” - “disulfate” + “sulfate” is “phosphate." I don't follow. Are diphosphate and disulphate tokens? Does the tokenization procedure not split out the di? Or the "-ite" termination? Those are literally rule-based tokens in IUPAC nomenclature.
summary_of_the_review: I think the idea of using IUPAC names is intriguing, but neither the theoretical arguments not the empirical results (due to lack of benchmarks) are convincing.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method C5T5, a self-supervised pre-training method based on the T5 pre-trained model, which is able to make zero-shot select-and-replace edits to satisfy specific property values. The specific difference of this paper is the IUPAC names (a standardized molecular representation), and the method is totally self-supervised. The experiments are evaluated on octanol-water partition, distribution coefficients, polar surface area, and refractivity, above four properties. Experiments show that the designed methods are able to achieve the optimization objective. 
main_review: The summary of the paper is concluded in the above contents. Here are the strengths and weaknesses of the paper from my view. 

Strengths:
1. The first attractive point is the IUPAC names representation method. As mentioned in the paper, this method is a standardized molecular representation that encodes the rich structural information but that has been largely ignored. The common approach of the representation for the molecule is the SMILES or graph. As introduced, this representation can handle more semantic information, and also the interpretability is better. 
2. The method is straightforward and simple, which uses a pre-training method T5 to model a conditional language model, where the condition is the property value. As a result, C5T5 does not require paired dataset for the molecule property improvement. 
3. The results on properties show the desired optimization is satisfied, and the different studies give a more clear understanding.

weaknesses:
Though I personally like the simple idea, there are still concerns and questions from my point,
1. The first concern is about the specific property value distribution. As shown in Table 5, each property only splits into three buckets. Then each model can only have three different types of property, which makes the model to be limited. For example, it can not clearly generate a better molecule if the initial molecule with logP is larger than 5.6. 
2. The second concern is about the IUPAC names. Though I feel happy to see a new representation method that seems to be better than other methods, the comparison is not clear and there seems no strong evidence or numbers can be shown in the paper. In this way, this can not clearly convince the superiority of IUPAC. However, I am pretty interested to see the goodness of IUPAC.
3. One specific question is about the pre-trained T5 model. The authors mentioned that they do not use sentencepiece, and only keep the IUPAC names as the vocabulary, which is different from the pre-trained embedding of T5 model. It is said that the first 1274 embeddings of English pre-trained embedding are kept. I am not sure about this process, what is the difference between this method and directly replacing the vocabulary with the 1274 new randomly initialized embeddings? Since the vocabulary is different at all. 
4. It is required to make a result comparison to see the advantage of this method. Otherwise, it is hard to evaluate. 
5. Other concerns have been discussed by the authors, for example, the training cost is high; it is not always possible to generate valid and satisfying molecules since there are no constraints in the modeling. 


summary_of_the_review: I personally feel good about this work, but unclear parts are main concerns and there are no comparison with other works.
review_title: null

Meta-review: This work proposes to use a transformer model and language model inspired self-supervised training techniques to generate local modifications of organic molecules. The use of IUPAC names coupled with language inspired pre-training is indeed an interesting idea worthy of exploration. The paper has a lot of promises in this regard but needs more work to deliver it through the finish line. In the rebuttal, the authors have provided strong arguments toward the advantages of using IUPAC representation. While these arguments make sense, they are more or less conceptual and better and more clear empirical evidences are required to back them up.
Decision: Reject