Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: ##########################################################################
 
Summary: 

This paper proposes a method for the retrosynthesis prediction of polymers. A challenge in this problem is the lack of synthetic data for polymers. The method attempts to leverage models for small molecule retrosynthesis predictions (where there is more abundant data), as well as domain specific constraints derived from the chemistry of a particular class of polymerization reactions. The method is shown to outperform some baselines that are commonly used in small molecule retrosynthesis.

##########################################################################

Reasons for score: 

Overall, I vote for acceptance. I think the paper proposes a novel approach for polymer retrosynthesis that performs better than some of the baseline methods. However, I have some concerns, especially about the overall problem formulation of polymer retrosynthesis.

##########################################################################

Strengths:

*Paper is written clearly

*Interesting multistep method and applied constraints to convert a polymer repeat unit to monomers

*Some useful ablation studies

Weaknesses:

*Some concerns about the relevance of the overall problem formulation

*The method focuses on a particular type of polymerization reaction called condensation polymerization. Not sure how generalizable this method is to other common industrially relevant polymerization reactions, such as chain-growth type reactions 

*The evaluation dataset is very small (52 examples)

##########################################################################

Questions and other comments:

*I wonder how practically relevant the overall polymer retrosynthesis problem is, at least in the way that it is currently presented. In small molecule synthesis (eg in medicinal chemistry or natural product synthesis) there is a well-defined target molecule containing the properties that we want, so it is useful to perform retrosynthesis on the target molecular structure to obtain the step by step synthesis procedure that describes how to create the target molecule from precursor starting materials. However, in polymer synthesis, there is a much less defined target polymer structure (we have a distribution of different polymer structures), and the synthetic procedure required to create the target polymer in condensation polymerization is typically a single step mixing of the monomer building blocks. In my opinion, the problem of converting the unit polymer to monomers and subsequently to precursor starting materials makes sense (and this is the aspect of the work that is similar to typical small molecule retrosynthesis because the unit polymer and monomers are small molecules). But the polymer induction part of the modeling, where we start with a given polymer repeat unit and convert it to the unit polymer, makes much less sense because in a real use case why wouldn’t you just perform the analysis (eg designing a new polymer or performing retrosynthetic analysis) directly on the unit polymer or monomer building blocks, with the knowledge that the target polymer structure is essentially just a repeated form of the unit polymer structure.

*How important is the polymer induction step to find the possible end groups? My intuition is that there are not that many possible end groups for condensation polymerization. It would be interesting to see the distribution of the average number of unit polymer candidates for each of the 52 examples.

*The additional seq2seq baseline for monomer proposal in Figure 7, where the repeat unit is converted to the unit polymer using a very simple heuristic (by adding the most common end group pairs) shows pretty competitive performance compared to the proposed PolyRetro model, and I think this simple heuristic is actually a very reasonable addition to the baselines. 

*The PolyRetro model seems to have very marginal improvements over the PolyRetro-USPTO model, which seems to suggest a simple one step retrosynthesis model is already pretty good for the polymer induction step? 

*Any thoughts about how to obtain more data for future developments in this area?



review_title: Recommendation to accept

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper focuses on polymer retro synthesis problem. This is a novel problem and is very challenging because of the very small amounts of training data available (<100). They use reaction templates collected from small molecule reactions and formulate polymer retrosynthesis as a constrained optimization problem. The authors claim that this is the first learning method that takes constraints in polymer retrosynthesis problem.

The paper is well written and easy to follow. Overall I am positive about the paper as it proposes a new important tasks and provides a method that works well to tackle the task. The usage of reaction templates from small molecules as well as the ability to deal with with small amounts of training data is potentially useful and can serve as a model for few shot modeling in this domain for other tasks as well.

As for weaknesses,  I think the the baseline transformer performance of 0% top-50 recovery can be improved (and hence not a representation of true baseline). For example, can unsupervised pretraining be used in some way to improve its performance? Or can the small molecule reaction prediction data be used (aka in Molecule Transformer [1]) to seed the transformer model? 

[1] Schwaller et al: Molecular Transformer: A Model for Uncertainty-Calibrated Chemical Reaction Prediction

review_title: A nice paper that tackles an important problem

final_score: rating: 7: Good paper, accept
review_content: # summary #
This paper is concerning the retrosynthesis of polymerizations from monomer candidates. 
 
A polymer is a very long chain of unit polymers. Therefore conventional retrosynthesis models do not suit well for polymerization since they focus on applications on small molecules. 
The paper defines a problem of predicting a unit polymer and monomers, given a polymer repeat unit (Eq. 5). 
A unit polymer can be decomposed into a polymer repeat unit and end-groups. 
Thus the first component of the polyretro is a probabilistic proposal of the repeat-unit + end-groups using a one-step reaction model based on template reactions (Eq.6). 
The second component of the polyretro is to choose combinations of monomers to synthesize the polymer unit (Eq.7). 
Experimental validations reveal that the proposed polyretro algorithm can achieve 71% tope-30 recovery accuracy in unit polymer prediction, and 50% top-5 recovery accuracy for monomer prediction. 

# comments #
I have been working on the retrosynthesis for smaller molecules and enjoys the paper reading. 
I think this polymer challenge is quite new, and polymer synthesis is important for real applications. 

Overall, the paper is well written and readability is satisfactory. 
Problem definition of Eq. 5 seems straightforward but at the same time reasonable. 
Technical sections (Sec 4. and 5) are easy to read: finds little difficulties to understand the main idea. 

I cannot identify the technical novelty of the proposed polyretro method. 
In Sec 3.2, "we use Retro*", but which parts of the algorithm are borrowed from Retro* and which parts are not? 

Since this is a new challenge, it is difficult to assess the significance of the target recovery rates in experiments. 
The Transformer baseline completely fails to recover targets in the experiments. This result itself is important to confirm that the typical retrosynthesis model is incapable. 
However, such a ``weak baseline is not much informative to assess the significance of the proposed method. 
The paper can be further empowered if the paper is equipped with additional experiments on a baseline method that actually works (not zero recovery scores). It can be a non-ML approach or a commercial software with block-box algorithms. 

Minor issue:
In Fig. 1, the light pink of the polymer is invisible depending on the display/printer environments. 

# evaluations #
(++) First attempt of retrosynthesis for polymer reactions, polymers are important in practical applications

(+) Readability is satisfactory in general. 

(-) Technical novelty is unclear for me

(-) Significance of the target recovery scores is unclear to me
review_title: ICLR 2021 Conference Paper2253 AnonReviewer1

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary
- This paper aims at solving an important yet challenging task, the polymer retrosynthesis problem.
- This paper formulates the condensation polymerization as a constrained optimization with prior structural knowledge.
- This paper proposes a framework to solve the problem and demonstrates its effectiveness in a few-shot benchmark.

---

Strengths
- Although I'm not an expert in chemistry, I already know that retrosynthesis is a very important problem in academia and industry. Thus, polymer retrosynthesis also seems to be practically important.
- This paper well explains how to apply a single-step retrosynthesis model (trained in a reaction benchmark) into the polymer retrosynthesis problem. All sub-steps (except domain adaptation) are reasonable and easy to understand.

---

Concern (or Weakness) \#1: Domain adaptation
- The domain adaptation is emphasized throughout this paper. However, I think this adaptation is hard to provide a meaningful gain in general, (a) fundamentally in the few-shot setting and (b) practically as shown in Figure 4 (right). Details of (a) and (b) are described below.
  - (a) In the few-shot setting, the prior empirical distribution $\hat{p}(T)$ is often meaningless. This is because the reaction templates are very diverse, e.g., 40k reactions in USPTO-50k has 10k different templates, so the distribution is often very sparse, especially in the few-shot setting. In this case, optimizing Equation (6) cannot improve $p(u|r)$. In my opinion, the authors' choice, using atom- and bond-counting features, might alleviate this issue, however, I'm not sure what is the (chemical) meaning of this simplified distribution, and it can generalize to other polymer benchmarks.
  - (b) As illustrated in Figure 4 (right), the adaptation does not provide any gain in the monomer prediction even though it depends on $p(u|r)$ as described in the caption of Figure 4. It means that the stability constraint will filter out some wrong predictions in $p(u|r)$ from PolyRetro-USPTO. It is then hard to say that the domain adaptation is helpful for polymer retrosynthetic optimization (Definition 4).
- Since tempalte-based models are used and $u$ is determined when $T$ is given, the PolyRetro model with domain adaptation can be written as $p_\text{PolyRetro}(T|r)=\lambda p_\text{USPTO}(T|b-A-A-c) + (1-\lambda)\hat{p}(T)$ where $p_\text{USPTO}(T|b-A-A-c)$ is the pre-trained single-step model. Am I correct? The notations are somewhat confused because all density functions are denoted by $p$.
- Hyperparameter settings:
  - How to decide $\lambda$ without the validation set? I think one fold should be used for validation.
  - What $h$ is used for experiments? Is $h$ in the $\hat{p}(T)$ the same as used in Definition 4?

Concern (or Weakness) \#2: Methodology Novelty
- All sub-steps (except domain adaptation) are just filtering processes using a pre-trained single-step model. While the authors well-formulate the polymer retrosynthesis problem and well-explain the sub-steps, they seem somewhat straightforward. I am willing to give credit for this work because this is the first one (to my knowledge), but I am concerned about novelty in terms of methodology.

Concern (or Weakness) \#3: No qualitative study
- In retrosynthesis, top-k accuracy cannot reflect practical scenarios since multiple solutions can exist. Thus many papers often provide their success/failure cases and allow readers to evaluate them. In my opinion, such studies are critical in this field because it can provide additional insight into a chemist even though a prediction is wrong.

Concern (or Weakness) \#4: Baselines
- What is the support of $p(u|r)$? I'm curious about how RandomProposal works.
- I cannot be sure that RandomProposal and Transformer are good baselines. As I mentioned in the second concern, designing Poly Retro-USPTO is somewhat straightforward given the constraints of the proposed problem. Thus I think PolyRetro-USPTO is a more proper and stronger baseline.
- In the Appendix, there are stronger baselines than RandomProposal and Transformer. Why are they in the Appendix? I think they should be presented in the main paper.
- I want to know I correctly understand how mlp-retro and seq2seq-retro work. As far as I understand, they follow three sub-steps: (a) Given $r$, apply a pre-trained single-step retrosynthesis model (template-based mlp or template-free seq2seq) into $H-r-H$; (b) If the model obtains $H-B-b+c-C-H\rightarrow H-r-H$; (c) Then, the final monomers are $b-B-b$ and $c-C-c$ and the unit polymer is $b-B-C-c$. If I'm right, I wonder why mlp-retro is failed (because it is similar to PolyRetro-USPTO), and seq2seq-retro is not working for additional polymers as described in Appendix (because it just uses a pre-trained model). If I'm wrong, that can be a stronger baseline than RandomProposal/Transformer.
- Is Transformer trained using only one training fold from scratch?

---

Questions
- What is $\mathcal{R}_{m1,m2}$? I think its definition should be provided.
- Are the end-groups ($b$, $c$ in papers' definitions) of polymers often unknown? Or, are they not important? I wonder why only the repeat unit $r$ is given in the polymer retrosynthesis problem.
- Why the monomers should be symmetric? I think $b-B-b' + c'-C-c \rightarrow b-A-c + b'+c'$ can be another available candidate under the stability constraint (Definition 2).

---

Conclusion: While this paper focuses on an important problem, I have several concerns mentioned above, so  I think this paper is on the borderline.

review_title: Review

Meta-review: This paper proposes a novel problem of polymer retrosynthesis, and a method to solve it. The authors formally define the polymer retrosynthesis optimization problem as a constrained problem to identify the monomers and the unit polymer, with the recursive and stability constraints. Further, since the main challenge with polymer retrosynthesis is the extremely scarce training data, the authors propose a domain adaptation technique that can utilize a single-step retrosynthesis model trained on a large amount of data. The authors also use Retro* [Chen et al. 20] for synthesizability check of the monomers. The proposed method, PolyRetro, is validated against few naive baselines for top-k recovery performance, and is shown to outperform them. 

All reviewers found the problem of polymer retrosynthesis tackled to be important as well as novel, and the paper to be very well-written. However, all reviewers had a common concern on the limited technical novelty and meaningless baselines that makes it difficult to evaluate the significance of the results. Some of the reviewers were also concerned with the insignificant performance gain with the proposed domain adaptation technique (PolyRetro vs. PolyRetro-USPTO in Figure 4), and its limited applicability to a condensation polymerization. The authors provided new results with more baselines, which fine-tune the single-step retrosynthesis model (MLP, seq-to-seq) trained on USPTO.

The below is the summary of pros and cons:

**Pros**
- The tackled polymer retrosynthesis problem is novel and practically important. 
- The proposed problem formulation and constraints are interesting and make sense.
- The paper is well-written and easy to follow even for non-domain experts.

**Cons**
- The proposed solution with recursive and stability constraints is rather straightforward, as well as the use of Retro* for screening out the monomers. 
- The domain adaptation technique, which is advertised as an important contribution to combat extreme data scarcity, is both straightforward and yields small performance gain.
- The baselines in the original version of the paper are simply meaningless strawmans, and the new baseline (seq2seq-retro) in Section D of the Appendix seems quite strong, making it difficult to validate the effectiveness of the proposed method.

The paper received split reviews, with three leaning toward acceptance and one leaning toward rejection. After the interactive discussion period with the authors, the reviewers had an in-depth discussion, where all reviewers agreed that the technical novelty or contribution to the general machine learning field, or general applicability to polymer synthesis is limited. The reviewers did not reach a consensus, which makes the paper a borderline case, and after the discussion with the program chairs, we decided to reject the paper due to the unresolved concerns.   

I believe that the proposed problem-specific solution is adequate, although it has little technical novelties, since this is an application paper. However what is more problematic is the inconclusive experimental validation results due to lack of meaningful baselines. I suggest the authors to compare against seq2seq retro + Retro* in order to properly validate the effectiveness of the proposed method. Also, results in Figure 7, or the polymerization examples in Section A of the appendix should be incorporated into the main paper. I also suggest that the authors drop domain adaptation from the title since it constitutes a small part of the method and thus is misleading. 
Decision: Reject