Paper: Towards 3D Molecule-Text Interpretation in Language Models (ICLR-2024)

Reviewer comment (Evaluation):
The paper misses to include some failure cases (e.g., hallucination) of the proposed method. For instance, in Table 3(b), it seems the proposed 3D-MoLM wrongly interprets Globostellatic acid B (C34H48O6 while the ground truth is C33H48O7); the paper needs to discuss when the model tends to predict the wrong results.

Author response:
**Response:** Thank you for your insightful suggestion. We agree that discussing failure cases is crucial for a comprehensive understanding of our model's performance. In response, we have revised our submission to include two failure case studies in Appendix D. Specifically, we show two mistakes of 3D-MoLM: 1) it confuses between Globostellatic acid C and Globostellatic acid B; 2) it fails to generate a molecule's correct chemical formula. We attribute these two mistakes to the following limitations:

1. The discernment of 3D-MoLM at a more refined granularity requires further enhancement. 
2. Existing language models, including GPT-4, are not yet fully capable of accurately counting the number of atoms and providing the correct chemical formula.

These cases underscore areas where 3D-MoLM can be improved. Thank you again for your constructive critique, which has significantly contributed to enhancing the quality of our paper.

Reviewer comment (Experiments):
Have the authors tried using larger model (e.g., LLaMA-13B)? If the authors tried, does it show the better performance?

Author response:
**Response:** Thank you for your suggestion. To address the question of how a language model's scale influences performance. we have included new experiments (Table 1 above) using MolT5-small (82M), MolT5-base (252M) and MolT5-large (782M). See our response to your Q2 for details. We can obseve that increasing LM's scale will also increase molecule captioning performance, which aligns with the common understanding in LM studies. 

While we recognize the potential advantages of employing a larger LM (e.g., Llama-13B), our current study is constrained by limited computational resouces. We appreciate your understanding and will certainly consider this in our future work as resources permit.

Reviewer comment (Novelty):
The novelty of this work is actually hard to say enough. It is clear that the framework is almost the same as BLIP2, the pre-training tasks and the fine-tuning stages are also the same.

Author response:
We appreciate your comments, but wish to respectfully emphasize our novelties and contributions. Our adaptation of Q-Former to 3D molecule-to-text generation tasks is nontrivial, offering a new dimension to 3D molecule understanding. 

Further, we have made significant modifications to BLIP-2 for adaptation to the molecule domain, and devise a dataset for 3D-molecule centric instruction tuning. Our novelties and contributions, compared to BLIP-2, are summarized below:

* **First 3D molecule-text modeling method.** 3D-MoLM is, to our knowledge, the first method to explore 3D molecule-text modeling. We believe 3D-MoLM advances the field by enabling text-based understanding of 3D molecules.
* **Multi-task Generalist Model through Instruction-tuning.** Unlike BLIP-2, 3D-MoLM includes a multi-modal instruction-tuning stage. This instruction tuning phase aims to align 3D-MoLM to human preference, and develop a generalist model that can perform various downsteam tasks. The performance of the generalist 3D-MoLM are reported in our updated Table 3 and Table 4. See our response to your Q4 for more details. 
* **Instruction-tuning Dataset.** We present 3D-MoIT for 3D molecule-text instruction tuning. 3D-MoIT is, to our knowledge, the first 3D molecule-centric instruction tuning dataset. We also propose the GPT-3.5 enriched version of PubChem for 3D molecule-text alignment. We believe these datasets can facilitate further research and development in this area. 

We hope this clarifies the unique contributions of our work and differentiate it from existing models like BLIP2.

Reviewer comment (Presentation):
In Figure 3, stage 1 is molecule-text retrieval, and stage 2 is the molecular caption, but in section 2.2.1, stage 1 describes multitask training with the whole three tasks. See "Following BLIP,m we perform multi-task training, including....". Please clarify the correct one.

Author response:
Thank you for identifying the ambiguities. We appreciate the opportunity to clarify the training stages. Stage 1 includes three training objectives: molecule-text matching, molecule-text contrasting, and molecule captioning. These objectives collectively endows the Q-Former with strong molecule-text retrieval ability. Consequently, upon completing stage 1, Q-Former is fine-tuned for the retrieval task. The term "molecule-text retrieval" in Figure 3 originally refers to stage 1's downstream task, rather than a specific training objective. 

To improve the clarify of our submission, we have made the following revisions:

* **Terminology Consistency.** We have refined the use of the term "task" throughout the paper. Initially, it ambiguously referred to both downstream applications and training objectives. After revision, "task" strictly denotes downstream applications, while "training objective" is used for training objective.
* **Figure 2 & 3 Updates.** Specifically, Figure 2b now illustrates the objectives and architectures of stage 2 & 3; Figure 3 uses consistent names for the three training stages across the paper.
* **Section 2.2.1 Revision.** The description of Stage 2 is now more explicit about its training objective.

We hope these revisions adequately address your concerns and improve the clarity of our paper. Thank you again for your meticulous feedback, which has been instrumental in improving our work.

Reviewer comment (Experiments):
For the downstream tasks, the model is fine-tuned after different stages, which causes multiple different models, each task for a model. I am wondering about the multitasking fine-tuning and the performance of one model. Or at least, after three/two stage trainings, then finetune on the downstream tasks.

Author response:
Thank you for your insightful comments. To address your concern, we have included the performances of both the specialist models, which are fine-tuned for each task individually, and the generalist model, which is a unified model trained on all tasks and evaluated using a consistent checkpoint. The performance is shown in the Table below:

**Table2: One checkpoint for captioning, open-text QA and computed properties.****

**Table 2a: Molecule captioning performance.**

|            | BLEU-2 | BLEU-4 | ROUGE-1 | ROUGE-2 | ROUGE-L | METEOR |
| ---------- | ------ | ------ | ------- | ------- | ------- | ------ |
| Specialist | 30.32  | 22.52  | 36.84   | 22.32   | 31.23   | 33.06  |
| Generalist | 29.25  | 22.07  | 36.48   | 21.80   | 30.95   | 33.12  |

**Table 2b: Descrptive open-text QA performance.**

|            | BLEU-2 | BLEU-4 | ROUGE-1 | ROUGE-2 | ROUGE-L | METEOR |
| ---------- | ------ | ------ | ------- | ------- | ------- | ------ |
| Specialist | 32.00  | 26.13  | 40.13   | 25.55   | 34.64   | 52.15  |
| Generalist | 31.61  | 25.88  | 39.93   | 25.68   | 34.78   | 51.74  |

**Table 2c: Computed properties QA performance.**

|            | Weight | LogP | TPSA  | Complexity | HOMO | LUMO | H-L Gap | SCF  |
| ---------- | ------ | ---- | ----- | ---------- | ---- | ---- | ------- | ---- |
| Specialist | 14.79  | 0.66 | 9.71  | 44.85      | 0.26 | 0.25 | 0.28    | 0.35 |
| Generalist | 16.58  | 0.78 | 10.90 | 45.49      | 0.35 | 0.36 | 0.32    | 0.38 |

The results show that while the generalist model slightly underperforms compared to the specialist models, it still demonstrates a performance gain over other baselines. This highlights its versatility and ability to handle multiple tasks effectively. Please refer to the performance tables (Table 3 & Table 4) in the revised paper and the training details in Appendix C for more information.

Reviewer comment (Experiments):
At the beginning of section 3.1, the authors mentioned "we assess the stage-1 checkpoint of ...", but the last sentence on page 6 is "Q-former benefits from the multi-task pre-training", if the stage only contains retrieval task, the claim is wrong.

Author response:
Thank you for identifying the ambiguities. We appreciate the opportunity to clarify the training stages. Stage 1 includes three training objectives: molecule-text matching, molecule-text contrasting, and molecule captioning. These objectives collectively endows the Q-Former with strong molecule-text retrieval ability. Consequently, upon completing stage 1, Q-Former is fine-tuned for the retrieval task. The term "molecule-text retrieval" in Figure 3 originally refers to stage 1's downstream task, rather than a specific training objective. 

To improve the clarify of our submission, we have made the following revisions:

* **Terminology Consistency.** We have refined the use of the term "task" throughout the paper. Initially, it ambiguously referred to both downstream applications and training objectives. After revision, "task" strictly denotes downstream applications, while "training objective" is used for training objective.
* **Figure 2 & 3 Updates.** Specifically, Figure 2b now illustrates the objectives and architectures of stage 2 & 3; Figure 3 uses consistent names for the three training stages across the paper.
* **Section 2.2.1 Revision.** The description of Stage 2 is now more explicit about its training objective.

We hope these revisions adequately address your concerns and improve the clarity of our paper. Thank you again for your meticulous feedback, which has been instrumental in improving our work.

Reviewer comment (Evaluation):
From Table 4, it seems the performance gain is not as large as expected on open-text QA, do the authors have some analysis? 2D is similar to 3D.

Author response:
Thank you for your comments. Table 4 has two subtables: Table 4a is about Descriptive Property QA, and Table 4c is about Computed Property QA. We now provide clarifications for the subtables separately:

1. **Table 4c (computed property QA).** For tasks related to predicting 3D-dependent properties, 3D-MoLM significantly outperforms baselines that uses 1D or 2D molecular representations, demonstraing 3D-MoLM's effectiveness in perceiving 3D molecular geometries.
2. **Table 4a (descriptive property QA)**. The QAs in this dataset are generated based on descriptions in PubChem, where only a small fraction of description is 3D-dependent. The majority QAs from PubChem can be answered by reading 1D or 2D molecular representations. This could account for the smaller improvement of using 3D perception for this dataset. 
3. **Table 4a (descriptive property QA).** Although the improvement for this dataset is smaller, 3D-MoLM still surpasses baselines. This demonstrates the effectiveness of 3D molecule-text interpretation even in tasks that may not have strong connection with 3D structures. 

We hope this explanation provides a clearer understanding of the performance of 3D-MoLM in different tasks.

Reviewer comment (Reproducibility):
In Appendix C, did the retrieval task use LORA for fine-tuning?

Author response:
Thanks for the question. No, we do not use LoRA for retrieval task, and the retrieval task uses checkpoint after the stage 1 directly, without additional fine-tuning. 

We have revised Figure 2 to illustruate 3D-MoLM’s architectures at different stages and revised Appendix C to include more experimental details. To clarify, the retrieval task includes only the Q-Former and the Uni-Mol encoder from training stage 1, without the Llama-2 model. Due to the smaller scales of Uni-Mol and Q-Former, we do not employ LoRA in training stage 1.

Reviewer comment (Novelty):
The motivation and needs for molecule-language multimodal models remain unclear. While multimodality between molecule and language domains is a promising research direction, the paper does not sufficiently discuss in what scenarios the proposed model can be utilized and how it could lead to the synthesis or retargeting of novel or existing compounds.

Author response:
Thank you for the insightful suggestion. We believe studying molecule-text multi-modal learning have values for scientists in (but not limited to) two aspects: 1) solving scientific tasks that are multi-modal in nature; 2) improving uni-modal scientific tasks by leveraging the science knowledge or reasoning ability of LMs. We now elaborate:

* **Solving multi-modal scientific tasks.** These tasks are multi-modal in nature and cannot be solved without multi-modal methods. For example, experimental procedure prediction [1] requires reading chemical reactions (molecule modality) and output experimental instructions (text modality). **This task has significant practical values for automating chemical synthesis experiment design**, and 3D-MoLM can be a foundation model for fine-tuning on this task. Another example is molecule-text retrieval, where chemists can specify desired chemical properties (e.g., biological activity and toxicity) using natural language, and retrieve molecules from database that includes unannotated molecules. **This task can help retargeting existing molecules.** 

* **Improving uni-modal scientific tasks.** Uni-modal tasks (e.g., molecule property prediction) can potentially be improved by using the chemistry knowledge in LMs. [2] shows an example where joint multi-modal molecule-text learning improves molecule property prediction performance. [3] shows an example where joint multi-modal learning between an LM and a protein encoder improves the protein encoder's protein classification performance.

Reviewer comment (Evaluation):
The specific use cases of the proposed molecule-language model are not described. It is unclear how scientists or practitioners (domain experts) would be able to harness the proposed method in their research or business.

Author response:
Thank you for the insightful suggestion. We believe studying molecule-text multi-modal learning have values for scientists in (but not limited to) two aspects: 1) solving scientific tasks that are multi-modal in nature; 2) improving uni-modal scientific tasks by leveraging the science knowledge or reasoning ability of LMs. We now elaborate:

* **Solving multi-modal scientific tasks.** These tasks are multi-modal in nature and cannot be solved without multi-modal methods. For example, experimental procedure prediction [1] requires reading chemical reactions (molecule modality) and output experimental instructions (text modality). **This task has significant practical values for automating chemical synthesis experiment design**, and 3D-MoLM can be a foundation model for fine-tuning on this task. Another example is molecule-text retrieval, where chemists can specify desired chemical properties (e.g., biological activity and toxicity) using natural language, and retrieve molecules from database that includes unannotated molecules. **This task can help retargeting existing molecules.** 

* **Improving uni-modal scientific tasks.** Uni-modal tasks (e.g., molecule property prediction) can potentially be improved by using the chemistry knowledge in LMs. [2] shows an example where joint multi-modal molecule-text learning improves molecule property prediction performance. [3] shows an example where joint multi-modal learning between an LM and a protein encoder improves the protein encoder's protein classification performance.

Reviewer comment (Theory):
Regarding the question-answering task, the authors performed molecular properties prediction, including HOMO/LUMO, which are typically derived from density functional theory calculations or molecular representation models such as Uni-Mol. There is no clear rationale or evidence provided for how the language model might enhance the prediction accuracy of these properties.

Author response:
Thank you for your insightful question. To address your concern, we have included an additional experiment where we fine-tuned one Uni-Mol model for eight computed property prediction tasks. As shown in the Table below, 3D-MoLM achieves comparable performance with Uni-Mol and outperforms it in predicting molecular weight, LogP, complexity, and SCF energy. We provide the following analysis to explain this:

**Table: One model trained for all properties. Lower values are better.**

|         | Weight    | LogP     | TPSA      | Complexity | HOMO     | LUMO     | H-L Gap  | SCF      |
| ------- | --------- | -------- | --------- | ---------- | -------- | -------- | -------- | -------- |
| Uni-Mol | 20.35     | **0.59** | 13.48     | 57.24      | **0.32** | **0.35** | **0.21** | 0.45     |
| 3D-MoLM | **16.58** | 0.78     | **10.90** | **45.49**  | 0.35     | 0.36     | 0.32     | **0.38** |

**LM's Chance to Improve.** LMs can potentially enhance the accuracy of molecular property prediction by leveraging the rich contextual knowledge found in chemistry literature. For instance, the pretraining corpus of 3D-MoLM contains descriptions about hydrophobicity (LogP) and solubility (TPSA). While Uni-Mol excels at predicting molecular properties by interpreting 3D geometric data, it cannot utilize textual descriptions of chemical properties, which are abundant in chemistry literature. By aligning the representations from Uni-Mol with an LM, the LM can leverage both the geometric knowledge from Uni-Mol and the rich contextual knowledge found in chemical literature. This dual-source knowledge utilization can enhance the prediction of molecular properties.

As per your suggestion, we have included Uni-Mol as a baseline in revised Table 4c. However, we would like to respectfully highlight the versatility of 3D-MoLM: 3D-MoLM extends beyond computed properties prediction, and can do captioning (Table 3) and open-text QA (Table 4a) that Uni-Mol cannot perform. This versatility is a significant advantage of our model, demonstrating the potential of multimodal learning in this domain.

Reviewer comment (Experiments):
The study only compared prediction accuracy with language models, but the ability to predict these properties comes from Uni-Mol. Therefore, Uni-Mol should also be included as a baseline in the experiments to demonstrate the effect of multimodal learning.

Author response:
Thank you for your insightful question. To address your concern, we have included an additional experiment where we fine-tuned one Uni-Mol model for eight computed property prediction tasks. As shown in the Table below, 3D-MoLM achieves comparable performance with Uni-Mol and outperforms it in predicting molecular weight, LogP, complexity, and SCF energy. We provide the following analysis to explain this:

**Table: One model trained for all properties. Lower values are better.**

|         | Weight    | LogP     | TPSA      | Complexity | HOMO     | LUMO     | H-L Gap  | SCF      |
| ------- | --------- | -------- | --------- | ---------- | -------- | -------- | -------- | -------- |
| Uni-Mol | 20.35     | **0.59** | 13.48     | 57.24      | **0.32** | **0.35** | **0.21** | 0.45     |
| 3D-MoLM | **16.58** | 0.78     | **10.90** | **45.49**  | 0.35     | 0.36     | 0.32     | **0.38** |

**LM's Chance to Improve.** LMs can potentially enhance the accuracy of molecular property prediction by leveraging the rich contextual knowledge found in chemistry literature. For instance, the pretraining corpus of 3D-MoLM contains descriptions about hydrophobicity (LogP) and solubility (TPSA). While Uni-Mol excels at predicting molecular properties by interpreting 3D geometric data, it cannot utilize textual descriptions of chemical properties, which are abundant in chemistry literature. By aligning the representations from Uni-Mol with an LM, the LM can leverage both the geometric knowledge from Uni-Mol and the rich contextual knowledge found in chemical literature. This dual-source knowledge utilization can enhance the prediction of molecular properties.

As per your suggestion, we have included Uni-Mol as a baseline in revised Table 4c. However, we would like to respectfully highlight the versatility of 3D-MoLM: 3D-MoLM extends beyond computed properties prediction, and can do captioning (Table 3) and open-text QA (Table 4a) that Uni-Mol cannot perform. This versatility is a significant advantage of our model, demonstrating the potential of multimodal learning in this domain.

Reviewer comment (Experiments):
The PubChem database provides the 3D conformation of each molecule, but it is unclear what method was used to obtain these structures. It may be worthwhile to consider leveraging PubChem's 3D conformation data instead of MMFF-relaxed structures.

Author response:
Thank you for your suggestion regarding the use of PubChem's 3D conformations. It is important to note that both PubChem [4] and RDKit [5] use MMFF to obtain 3D conformations, although their MMFF implementations have subtle differences. To account for this difference, we have added PubChem's 3D conformations to our dataset, as an alternative to RDKit-generated 3D conformations. We plan to make both available online. The statistics of PubChem3D conformations are show below. 

**Table: The coverage of PubChem3D's 3D conformations on our PubChem dataset.**

|          |         | PubChem3D       | RDKit-generated |
| -------- | ------- | --------------- | --------------- |
| Subset   | Size    | Cover Ratio (%) | Cover Ratio (%) |
| pretrain | 309,689 | 67.2            | 97.4            |
| train    | 12,000  | 61.8            | 100.0           |
| valid    | 1,000   | 63.3            | 100.0           |
| test     | 2,000   | 64.3            | 100.0           |

We can observe that PubChem provides 324,689 molecules with textual descriptions in total. Yet, only about two thirds of them are paired with 3D information. RDKit-generated conformation covers 100% downstream molecules because our pre-processing for downstream sets has filtered out molecules that fail to pass RDKit's MMFF algorithm.

Reviewer comment (Evaluation):
The analysis of the proposed 3D-MoLM is not enough.

Author response:
We sincerely thank you for your valuable comments and positive feedback! They have helped us to enrich the analysis of 3D-MoLM for a better understanding. To address your concern, we present the following response.

> **Q1: More insight and analysis.** The analysis of the proposed 3D-MoLM is not enough. The proposed method is interesting and straightforward, any insight or analysis that could be offered to read to have a good understanding?

**Response:** Thank you for your constructive feedback. We agree that an in-depth analysis can help readers better understand 3D-MoLM's performances, and identify potential directions for future research and improvement.

**Firstly,** we have updated our paper to include a comparison between the performance of specialist 3D-MoLMs, which are fine-tuned for each task individually, and the generalist 3D-MoLM, which is a unified model trained on all tasks and evaluated using a consistent checkpoint. We also summarize the performance of them in the table below:

**Table1: One checkpoint for captioning, open-text QA and computed properties.****

**Table 1a: Molecule captioning performance.**

|            | BLEU-2 | BLEU-4 | ROUGE-1 | ROUGE-2 | ROUGE-L | METEOR |
| ---------- | ------ | ------ | ------- | ------- | ------- | ------ |
| Specialist | 30.32  | 22.52  | 36.84   | 22.32   | 31.23   | 33.06  |
| Generalist | 29.25  | 22.07  | 36.48   | 21.80   | 30.95   | 33.12  |

**Table 1b: Descrptive open-text QA performance.**

|            | BLEU-2 | BLEU-4 | ROUGE-1 | ROUGE-2 | ROUGE-L | METEOR |
| ---------- | ------ | ------ | ------- | ------- | ------- | ------ |
| Specialist | 32.00  | 26.13  | 40.13   | 25.55   | 34.64   | 52.15  |
| Generalist | 31.61  | 25.88  | 39.93   | 25.68   | 34.78   | 51.74  |

**Table 1c: Computed properties QA performance.**

|            | Weight | LogP | TPSA  | Complexity | HOMO | LUMO | H-L Gap | SCF  |
| ---------- | ------ | ---- | ----- | ---------- | ---- | ---- | ------- | ---- |
| Specialist | 14.79  | 0.66 | 9.71  | 44.85      | 0.26 | 0.25 | 0.28    | 0.35 |
| Generalist | 16.58  | 0.78 | 10.90 | 45.49      | 0.35 | 0.36 | 0.32    | 0.38 |

The results show that while the generalist model slightly underperforms in comparison to the specialist models, it still exhibits a performance gain over other baselines. This underscores its versatility and capability to effectively handle multiple tasks. For more detailed information, please refer to the performance tables in the revised paper (Table 3 and Table 4 in the revised paper) and the training details in Appendix C.

**Secondly,** we have revised our draft to include failure case studies in Appendix D, serving as preliminary examinations of 3D-MoLM's limitations. We identify two main limitations:

* 3D-MoLM's ability to discern fine-grained small molecular structures can be improved. In the revised Table 6, we show an example that 3D-MoLM confuses between Globostellatic acid C and Globostellatic acid B, where the only difference is the position of a methoxy group.

* We show that 3D-MoLM, and also GPT-4, cannot accurately count the number of atoms and generate the chemical formulas.

We are committed to continuously refining 3D-MoLM to address these issues. For a more detailed analysis and failure case study, please refer to Appendix D in the revised paper. We hope this analysis provides a more nuanced understanding of 3D-MoLM's capabilities and prompts future enhancement.

Reviewer comment (Writing):
The proposed method is interesting and straight-forward, any insight or analysis that could be offered to read to have a good understanding?

Author response:
We sincerely thank you for your valuable comments and positive feedback! They have helped us to enrich the analysis of 3D-MoLM for a better understanding. To address your concern, we present the following response.

> **Q1: More insight and analysis.** The analysis of the proposed 3D-MoLM is not enough. The proposed method is interesting and straightforward, any insight or analysis that could be offered to read to have a good understanding?

**Response:** Thank you for your constructive feedback. We agree that an in-depth analysis can help readers better understand 3D-MoLM's performances, and identify potential directions for future research and improvement.

**Firstly,** we have updated our paper to include a comparison between the performance of specialist 3D-MoLMs, which are fine-tuned for each task individually, and the generalist 3D-MoLM, which is a unified model trained on all tasks and evaluated using a consistent checkpoint. We also summarize the performance of them in the table below:

**Table1: One checkpoint for captioning, open-text QA and computed properties.****

**Table 1a: Molecule captioning performance.**

|            | BLEU-2 | BLEU-4 | ROUGE-1 | ROUGE-2 | ROUGE-L | METEOR |
| ---------- | ------ | ------ | ------- | ------- | ------- | ------ |
| Specialist | 30.32  | 22.52  | 36.84   | 22.32   | 31.23   | 33.06  |
| Generalist | 29.25  | 22.07  | 36.48   | 21.80   | 30.95   | 33.12  |

**Table 1b: Descrptive open-text QA performance.**

|            | BLEU-2 | BLEU-4 | ROUGE-1 | ROUGE-2 | ROUGE-L | METEOR |
| ---------- | ------ | ------ | ------- | ------- | ------- | ------ |
| Specialist | 32.00  | 26.13  | 40.13   | 25.55   | 34.64   | 52.15  |
| Generalist | 31.61  | 25.88  | 39.93   | 25.68   | 34.78   | 51.74  |

**Table 1c: Computed properties QA performance.**

|            | Weight | LogP | TPSA  | Complexity | HOMO | LUMO | H-L Gap | SCF  |
| ---------- | ------ | ---- | ----- | ---------- | ---- | ---- | ------- | ---- |
| Specialist | 14.79  | 0.66 | 9.71  | 44.85      | 0.26 | 0.25 | 0.28    | 0.35 |
| Generalist | 16.58  | 0.78 | 10.90 | 45.49      | 0.35 | 0.36 | 0.32    | 0.38 |

The results show that while the generalist model slightly underperforms in comparison to the specialist models, it still exhibits a performance gain over other baselines. This underscores its versatility and capability to effectively handle multiple tasks. For more detailed information, please refer to the performance tables in the revised paper (Table 3 and Table 4 in the revised paper) and the training details in Appendix C.

**Secondly,** we have revised our draft to include failure case studies in Appendix D, serving as preliminary examinations of 3D-MoLM's limitations. We identify two main limitations:

* 3D-MoLM's ability to discern fine-grained small molecular structures can be improved. In the revised Table 6, we show an example that 3D-MoLM confuses between Globostellatic acid C and Globostellatic acid B, where the only difference is the position of a methoxy group.

* We show that 3D-MoLM, and also GPT-4, cannot accurately count the number of atoms and generate the chemical formulas.

We are committed to continuously refining 3D-MoLM to address these issues. For a more detailed analysis and failure case study, please refer to Appendix D in the revised paper. We hope this analysis provides a more nuanced understanding of 3D-MoLM's capabilities and prompts future enhancement.