Paper: Domain-Agnostic Molecular Generation with Chemical Feedback (ICLR-2024)

Reviewer comment (Experiments):
The experiment is incomplete. The authors should conduct more comprehensive experiments (i.e., baseline comparisons and ablation studies) to demonstrate the effectiveness of the proposed MOLGEN model.

Author response:
We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

**1. More baselines**

Our selection of baseline models included notable ones like LIMO, Chemformer, and RetMol from post-2022, chosen for their well-recognized effectiveness in this field. It's important to note that finding universally recognized and effective baseline models in this rapidly evolving field can be challenging, especially with some newer models lacking publicly available code or comprehensive documentation. According to your recommendation, we have now incorporated a new baseline model, RT[1], into our analysis. **(Highlighted in Table 4 Page 7, and Appendix E Page 18)**

**2. More ablation studies**

In response to your suggestions, we have added the results of these ablation studies to **Appendix H.5 (Highlighted in Page 23)**.

- **Effect of label smoothing**

We investigate the impact of label smoothing on the diversity of molecules generated by the model, employing IntDiv as the metric. IntDiv assesses the chemical diversity of generated molecules by calculating the average Tanimoto coefficient within the generated set. As shown in the following table, the model with label smoothing does not overly rely on singular, frequently occurring patterns learned from the training data. Consequently, this enhances the diversity and creativity of the molecules generated.

|Model|Synthetic|Natural Product|
|-|-|-|
|MolGen (w/o label smoothing)|.8556|.8769|
|MolGen|.8567 **(+0.12%)**|.8878 **(+1.24%)**|

- **Effect of $\alpha$ parameter**

We also explore the impact of the hyperparameter $\alpha$. As illustrated in the following tables, an $\alpha$ value of 0 indicates no self-feedback.  When $\alpha$ is increased to 3, the model demonstrates a superior ability to optimize molecules compared to an $\alpha$ of 1. However, increasing $\alpha$ to 5 does not necessarily lead to better performance than at an $\alpha$ of 3. During actual operation, it is necessary to adjust parameters according to specific requirements. Based on experience, setting $\alpha$ to either 3 or 5 is recommended. 

|P-logP|Synthetic|Natural Product|
|-|-|-|
|$\alpha$=0|-13.152|-7.031|
|$\alpha$=1|0.128|1.235|
|$\alpha$=3|0.569|1.762|
|$\alpha$=5|0.860|1.569|

|QED|Synthetic|Natural Product|
|-|-|-|
|$\alpha$=0|0.488|0.767|
|$\alpha$=1|0.532|0.802|
|$\alpha$=3|0.675|0.883|
|$\alpha$=5|0.811|0.795|

Reviewer comment (Experiments):
Baseline comparison: Most of the baseline models in the experiment are from before 2022 (e.g., JT-VAE (2018), GCPN (2018), GraphAF (2020), and GraphDF (2021)). The authors should compare with more recent baselines to validate MOLGEN.

Author response:
We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

**1. More baselines**

Our selection of baseline models included notable ones like LIMO, Chemformer, and RetMol from post-2022, chosen for their well-recognized effectiveness in this field. It's important to note that finding universally recognized and effective baseline models in this rapidly evolving field can be challenging, especially with some newer models lacking publicly available code or comprehensive documentation. According to your recommendation, we have now incorporated a new baseline model, RT[1], into our analysis. **(Highlighted in Table 4 Page 7, and Appendix E Page 18)**

Reviewer comment (Experiments):
Ablation study: The authors only performed an ablation study on the self-feedback mechanism. In Eq. 8, the authors used the "soft" label to smooth the one-hot distribution (Eq. 2) into the target distribution, which is one of the contributions of the paper. The effectiveness of the "soft" label also needs to be verified in the ablation study. Additionally, the hyper-parameter "$\alpha$" controls the balance of the two losses and has an important impact on model performance. The choice of $\alpha$ and the effect of different $\alpha$ values on performance should also be demonstrated in the experiments.

Author response:
We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

**2. More ablation studies**

In response to your suggestions, we have added the results of these ablation studies to **Appendix H.5 (Highlighted in Page 23)**.

- **Effect of label smoothing**

We investigate the impact of label smoothing on the diversity of molecules generated by the model, employing IntDiv as the metric. IntDiv assesses the chemical diversity of generated molecules by calculating the average Tanimoto coefficient within the generated set. As shown in the following table, the model with label smoothing does not overly rely on singular, frequently occurring patterns learned from the training data. Consequently, this enhances the diversity and creativity of the molecules generated.

|Model|Synthetic|Natural Product|
|-|-|-|
|MolGen (w/o label smoothing)|.8556|.8769|
|MolGen|.8567 **(+0.12%)**|.8878 **(+1.24%)**|

- **Effect of $\alpha$ parameter**

We also explore the impact of the hyperparameter $\alpha$. As illustrated in the following tables, an $\alpha$ value of 0 indicates no self-feedback.  When $\alpha$ is increased to 3, the model demonstrates a superior ability to optimize molecules compared to an $\alpha$ of 1. However, increasing $\alpha$ to 5 does not necessarily lead to better performance than at an $\alpha$ of 3. During actual operation, it is necessary to adjust parameters according to specific requirements. Based on experience, setting $\alpha$ to either 3 or 5 is recommended. 

|P-logP|Synthetic|Natural Product|
|-|-|-|
|$\alpha$=0|-13.152|-7.031|
|$\alpha$=1|0.128|1.235|
|$\alpha$=3|0.569|1.762|
|$\alpha$=5|0.860|1.569|

|QED|Synthetic|Natural Product|
|-|-|-|
|$\alpha$=0|0.488|0.767|
|$\alpha$=1|0.532|0.802|
|$\alpha$=3|0.675|0.883|
|$\alpha$=5|0.811|0.795|

Reviewer comment (Writing):
Model clarity sub-question: Does the "$l$" in the paragraph above Eq. (1) indicate the total number of SELFIES in the dataset? Is it the same as the "$l$" in "two sets of "$l$" tunable prefix vectors" above Eq. (3)?

Author response:
**3.  Clarify in Section 2**

We apologize for any confusion caused by the notation in our manuscript. 

- **The use of $l$:** To clarify, in Eq. (1), $l$ refers to the length of the SELFIES strings in the dataset. In contrast, the $l$ mentioned in the context of "two sets of $l$ tunable prefix vectors" above Eq. (3) actually refers to the length of the prefix vectors themselves. We have revised the manuscript, changing this latter instance to $m$ for clearer differentiation. **(Highlighted in Section 2.1, Page 3, Last paragrapgh)**

Reviewer comment (Experiments):
Model clarity sub-question: What do the two tunable prefix vectors refer to, and why are these two vectors used in multi-head? How to prove that the performance is improved after using them?

Author response:
**3.  Clarify in Section 2**

We apologize for any confusion caused by the notation in our manuscript. 

- **Purpose of prefix vectors:** The two tunable prefix vectors, $\boldsymbol{P}_k$ (prefix for keys) and $\boldsymbol{P}_v$ (prefix for values), serve as additional context that is prepended to the keys and values in each attention head. These prefixes allow the model to adapt its attention mechanism to capture broader, domain-level patterns that are not immediately apparent from the input sequences alone. 

- **Usage in multi-head attention:** In a multi-head attention mechanism, different heads can focus on different aspects of the input. The incorporation of these prefix vectors across all heads ensures that each aspect of attention computation, regardless of the focus of a particular head, is informed by the same baseline domain knowledge. This consistency is crucial for ensuring that the model's learning is not just specific to the nuances of the input data but is also aligned with broader domain-specific patterns.

- **Prove the performance:**

In response to your valuable suggestion, we have now incorporated an ablation study in our manuscript **(Highlighted in Appendix H.5, Page 24)**. To investigate the impact of prefix tuning on the model, we present the mean (and standard deviation) of penalized logP improvement for molecules generated compared to inputs under varying similarity constraints, as detailed in the following table.  This data demonstrates that prefix tuning enhances the model's performance in optimizing molecules. 

|Method|sim=0.6|sim=0.4|
|-|-|-|
|MolGen (w/o prefix)|11.63±0.18|10.23±1.47|
|MolGen|12.08±0.82 **(+0.45)**|12.35±1.21 **(+2.12)**|  

Additionally, **Figure 8 (Page 9)** provides a comparison of the model's attention to key substructures, both with and without prefix tuning. For enhanced clarity, we have also included a tabular representation of the data visualized in this figure, offering a more direct interpretation of the impact of prefix tuning.

|Model|SAL|
|-|-|
|SMILES-based PLM|0.396±0.054|
|MolGen(w/o prefix)|0.476±0.095 **(+20.2%)**|
|MolGen|0.495±0.105 **(+25.0%)**|

As indicated in the aforementioned tables, the implementation of domain-agnostic prefix tuning not only enables the model to more effectively adapt to downstream tasks but also improves its interpretability.

Reviewer comment (Writing):
In the sentence "Finally, we encode the corrupted SMILES using a bidirectional model..." above Eq. (1), should it refer to "SMILES" or "SELFIES"?

Author response:
**5. Other typos**

We are grateful for your attention to these details and pointing out these oversights. We have made the following revisions to our paper according to your suggestions:

- Corrected "SMILES" to "SELFIES" **(Highlighted in Section 2.1, Page 3)**.

Reviewer comment (Reproducibility):
How are the $(S_i, S_j)$ pairs in $\bf S^{*}$ selected?

Author response:
**4. Pair selected method**

In the candidate SELFIES set $\mathcal{S^*}$, every distinct pair of SELFIES forms a ($S_i$, $S_j$) pair. This means that for any two different SELFIES strings in $\mathcal{S^*}$, say $S_i$ and $S_j$, we consider them together as a pair for our analysis.

Reviewer comment (Presentation):
Some typos: Figures 2 and 3 have opposite locations.

Author response:
**5. Other typos**

We are grateful for your attention to these details and pointing out these oversights. We have made the following revisions to our paper according to your suggestions:

- Swapped the positions of Figure 2 and 3 **(Page 3)**.

Reviewer comment (Writing):
Some typos: It would be preferable to replace "ours" in tables 1, 2 and 3 with "MOLGEN". In addition, both "MOLGEN" and "MolGen" appear in the context, which should be consistent.

Author response:
**5. Other typos**

We are grateful for your attention to these details and pointing out these oversights. We have made the following revisions to our paper according to your suggestions:

- Replaced the term "ours" with our model's specific name, "MOLGEN" **(Table 1-3)**.
- Ensured that the name of our model is consistently spelled as "MOLGEN" in all figures and context **(Figure 6-8, and Appendix Figure 2, 5)**.

Reviewer comment (Writing):
Some typos: The masking and start tokens are denoted by "[MASK]" and "[S]" in the context, but are "[Mask]" and "[s]" in Figure 3. Consistency is needed.

Author response:
**5. Other typos**

We are grateful for your attention to these details and pointing out these oversights. We have made the following revisions to our paper according to your suggestions:

- Revised "[Mask]" to "[MASK]" in Figure 2 **(Page 3)**.

Reviewer comment (Writing):
The authors introduce the notion of "molecular hallucination" as the generation of molecules that do not exhibit the desired properties or follow human preferences. The term appears to be a misnomer, as it does not align with the traditional understanding of "hallucination" in the NLP domain. In NLP literature, hallucination generally refers to the generation of fictitious or nonexistent entities or events [1,2,3]. Conversely, the authors assert that their "hallucinated" molecules are chemically valid, meaning they are neither fictitious nor nonexistent. Therefore, the use of the term "hallucination" could be misleading and should be reconsidered for clarity and conceptual consistency.

Author response:
We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

**1. Concept of molecular hallucination**

We apologize for any ambiguity in our expression, but we wish to clarify that our concept of "molecular hallucination" is indeed aligned with the NLP concept of hallucination. In NLP, hallucination refers to generating text or responses that, while grammatically correct, fluent, and natural, deviate from the provided source inputs (faithfulness) or lack factual accuracy (factualness) [1] [2] [3] [4].

Therefore, in our study, "molecular hallucination" is used to describe the phenomenon where molecules created by language models, although chemically valid and adhering to basic structural rules (similar to producing grammatically correct sentences), fail to possess the anticipated chemical properties (akin to producing text that deviates from the intended message or content of the source input). This concept is consistent with the understanding of hallucination in NLP, and as you may have noticed, other reviewers have also concurred with this viewpoint.

To enhance clarity, we have included related works about hallucination and further clarified the concept of "molecular hallucination" **(Highlighted in Appendix D.3)**.

Reviewer comment (Novelty):
The related works on hallucination are missing.

Author response:
We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

**1. Concept of molecular hallucination**

We apologize for any ambiguity in our expression, but we wish to clarify that our concept of "molecular hallucination" is indeed aligned with the NLP concept of hallucination. In NLP, hallucination refers to generating text or responses that, while grammatically correct, fluent, and natural, deviate from the provided source inputs (faithfulness) or lack factual accuracy (factualness) [1] [2] [3] [4].

Therefore, in our study, "molecular hallucination" is used to describe the phenomenon where molecules created by language models, although chemically valid and adhering to basic structural rules (similar to producing grammatically correct sentences), fail to possess the anticipated chemical properties (akin to producing text that deviates from the intended message or content of the source input). This concept is consistent with the understanding of hallucination in NLP, and as you may have noticed, other reviewers have also concurred with this viewpoint.

To enhance clarity, we have included related works about hallucination and further clarified the concept of "molecular hallucination" **(Highlighted in Appendix D.3)**.

Reviewer comment (Presentation):
The x-axis of the two subfigures on the right of Figure 7 is unclear.

Author response:
We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

**4. Modifications to the figures**

We have revised these figures to better facilitate reader understanding.

- For **Figure 7 (Highlighted in Page 8)**, the x-axis represents the QED values, and the y-axis shows their distribution.
- For **Figure 8 (Highlighted in Page 9)**, the first subfigure has the x-axis representing the Substructure Attention Level (SAL) values, while the second subfigure's x-axis also represents SAL values and the y-axis shows their distribution.
- For **Appendix Figure 2 (Highlighted in Page 21)**, the x-axis represents the property values and the y-axis shows their distribution.

Reviewer comment (Presentation):
The x-axis and y-axis of the two subfigures on the right of Figure 8 are unclear.

Author response:
We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

**4. Modifications to the figures**

We have revised these figures to better facilitate reader understanding.

- For **Figure 7 (Highlighted in Page 8)**, the x-axis represents the QED values, and the y-axis shows their distribution.
- For **Figure 8 (Highlighted in Page 9)**, the first subfigure has the x-axis representing the Substructure Attention Level (SAL) values, while the second subfigure's x-axis also represents SAL values and the y-axis shows their distribution.
- For **Appendix Figure 2 (Highlighted in Page 21)**, the x-axis represents the property values and the y-axis shows their distribution.

Reviewer comment (Novelty):
This is not the first molecule language model pretrained using molecule's SELFIE representations. [4] is also pretrained using SELFIES. The relevance and difference to [4] should be discussed.

Author response:
**5. Related paper**

Thank you for bringing this to our attention. While it is true that this study [6] also uses molecule's SELFIES representations for pretraining, the focus and methodology differ significantly from our work. In the referenced paper, the approach reinterprets regression as a sequence modeling task, where training combines property tokens with text tokens. Their model is engineered to generate molecules by inputting expected molecular property values along with a given molecular scaffold (with the generated molecules incorporating this scaffold), or to predict molecular property values based on an input molecule.

In contrast, our model is specifically designed to generate entirely new molecules based on existing ones, without being confined to a predetermined scaffold. This key difference in our objectives and techniques distinctly sets our research apart from the approach outlined in the cited study. 

Following your suggestion, we have now included the referenced paper as a new baseline in **Table 4 (Highlighted in Page 7)** and a discussion of this study in **Appendix E (Highlighted in Page 18)** of our revised manuscripts.

Reviewer comment (Writing):
In the second paragraph of the introduction, the authors claim that “the brittleness of SMILES may lead to a high proportion of generated chemically invalid strings.” Are there any citations to support this claim?

Author response:
**6. Other revisions**

We are grateful for your attention to these details and pointing out these oversights. In response to your valuable suggestions, we have made the following revisions:

- **Brittleness of SMILES:** We added necessary references to support the statement about SMILES potentially leading to meaningless sequences. **(Highlighted in Section 1, Page 2)**

Reviewer comment (Evaluation):
What potential limitations might the self-feedback mechanism introduce in molecular generation?

Author response:
Thank you very much for your insightful comments. Below are our responses:

- **Limitations about self-feedback**

Currently, the self-feedback mechanism in our study is tailored for optimizing a single target objective. As we mentioned in **Appendix B**, this approach exhibits reduced efficacy when applied to scenarios involving multiple objectives. This is primarily due to the generation of divergent preference rankings, which leads to uncertainty in the model's decision-making process about which optimization directions to prioritize. Consequently, an area for future research could be the development of self-feedback techniques that are more adept at handling and optimizing multiple objectives simultaneously.

Thank you again for your constructive suggestions! We hope our responses address your concerns.

Reviewer comment (Experiments):
Although an ablation study was conducted to check the self-feedback paradigm, there was no ablation experiment carried out to assess the use of SELFIES over SMILES. Instead, only a comparison was made between Chemformer and MolGen. Technically, a SELFIES-based MolecularLM implemented on BART does not seem irreplaceable. Given that many works involving molecule generation are still based on SMILES (such as MoMu, MolT5), the effectiveness of SELFIES in the work lacks further experimental verification.

Author response:
**1. Effectiveness of SELFIES**

We apologize for any confusion that may have arisen from our presentation. 

- To further clarify, as detailed in Appendix F, we have conducted a comparative analysis using both SMILES and SELFIES formats. In **Table 1, Figure 6, and Figure 8**, Chemformer (S-PLM), which is pre-trained on SMILES and employs a BART architecture, serves as our baseline for comparison. This setup, with MolGen not utilizing the self-feedback mechanism, ensures an unbiased evaluation. The experimental results demonstrate that using SELFIES as the molecular language enables the generation of 100% valid molecules, while also facilitating a more focused attention mechanism in the model.
- The decision to use SELFIES is based on its standardized and more simplified syntax, which we find to be particularly well-suited for processing by language models. This choice is reflected in our results, where SELFIES ensured the generation of 100% valid molecules, aligning well with our study's objectives. We wish to underscore that SMILES remains a vital and widely adopted format for molecular representation, and our study does not intend to diminish its importance. Instead, our utilization of SELFIES represents a novel exploration, driven by the specific objectives of our research.

Reviewer comment (Experiments):
The significance of the 'domain-agnostic molecular prefix tuning' step is questionable. It seems to be merely a measure to avoid overfitting in the overall model. Whether synthetic molecule generation and natural product generation in drug discovery can be considered two different tasks, and whether other dataset partitioning methods would have similar effects, are not explained. Therefore, the comparison of molecular distribution learning in the paper lacks persuasiveness.

Author response:
**2. Synthetic molecule v.s. natural product generation**

The division of the molecular dataset into "synthetic" and "natural product" domains is to effectively explore and understand molecules of varying complexities and origins. The "synthetic" domain encompasses artificially synthesized chemical molecules tailored for specific needs, e.g., drug development. On the other hand, the "natural product" domain covers molecules naturally occurring, which are pivotal in biological activities and often provide insights for drug development. Natural product molecules generally exhibit greater structural complexity and diversity (further data statistics are available in Appendix C), often resulting from the myriad of unique chemical structures produced through natural biological processes. This classification helps us better understand the unique challenges and features of each domain. 

In our research, we follow the methodologies of prior works[1] for distribution learning, where the baselines focus on synthetic molecule generation. Building upon this foundation, we have extended our scope by including the generation of natural products as a new and more challenging task. This expansion not only enhances the complexity of the tasks we address but also broadens the applicability of our model to a wider range of molecular structures encountered in various scientific domains. Given that both synthetic and natural molecules represent a significant portion of molecules in daily life and research, we posit that our model’s proficiency in these domains indicates its capability to handle various molecular tasks effectively. In response to your suggestion, we have elaborated on this aspect in **Appendix C (Page 16)**.

**3. Effectiveness of prefix tuning**

We performed an additional experiment to investigate the impact of prefix tuning **(Highlighted in Appendix H.5, Page 24)**. Here, we present the mean (and standard deviation) of penalized logP improvement for molecules generated compared to inputs under varying similarity constraints, as detailed in the following table.  This data demonstrates that prefix tuning enhances the model's performance in optimizing molecules.

|Method|sim=0.6|sim=0.4|
|-|-|-|
|MolGen (w/o prefix)|11.63±0.18|10.23±1.47|
|MolGen|12.08±0.82 **(+0.45)**|12.35±1.21 **(+2.12)**|

Additionally, **Figure 8 (Page 9)** provides a comparison of the model's attention to key substructures, both with and without prefix tuning. For enhanced clarity, we have also included a tabular representation of the data visualized in this figure, offering a more direct interpretation of the impact of prefix tuning.

|Model|SAL|
|-|-|
|SMILES-based PLM|0.396±0.054|
|MolGen(w/o prefix)|0.476±0.095 **(+20.2%)**|
|MolGen|0.495±0.105 **(+25.0%)**|

As indicated in the aforementioned tables, the implementation of domain-agnostic prefix tuning not only enables the model to more effectively adapt to downstream tasks but also improves its interpretability.

Reviewer comment (Evaluation):
Given that molecule generation typically needs to cater to a variety of requirements, have the authors considered other metrics beyond penalized logP, QED, and binding affinity for two human proteins? More persuasive experiments addressing a broader range of molecular properties could significantly enhance the applicability and robustness of the proposed model.

Author response:
**4. More metrics beyond penalized logP, QED, and binding affinity**

In our study, we select penalized logP and QED due to their widespread recognition as standard benchmarks for evaluating simple molecular properties. Additionally, we include binding affinity to assess a more complex and practical property. We believe that the combination of these metrics effectively demonstrates our method's ability to generate molecules with both simple and complex desired properties, thereby reflecting the model's versatility and generalizability.

At this stage, while acknowledging the importance of a diverse range of metrics, we have chosen to focus on these core indicators in this work. We strongly agree with your perspective and are dedicated to identifying and integrating more practically valuable metrics in our future research endeavors, further enhancing the applicability and robustness of our model.

Reviewer comment (Presentation):
The correspondence between the attention scores and the specific molecular structures in Figure 8 and Appendix Figure 5 is not very intuitive. The current figures do not convincingly demonstrate that the SMILES-based PLM is focusing attention on less relevant positions. It would be beneficial if the authors could revise this figure to improve its clarity and interpretability, thereby aiding readers in better understanding the model's inner workings.

Author response:
**5. Improving the clarity of figures**

In response to your comments, we have thoroughly revised both **Figure 8 and Appendix Figure 5**. These updates aim to enhance the clarity and interpretability of the correlation between attention scores and specific molecular structures.

Thank you again for your valuable suggestions! Your input has greatly contributed to enhancing the completeness and readability of our paper.