Atomas: Hierarchical Adaptive Alignment on Molecule-Text for Unified Molecule Understanding and Generation
Authors: Yikun Zhang
Geyan Ye
Chaohao Yuan
Bo Han
Long-Kai Huang
Jianhua Yao
Wei Liu
Yu Rong
Venue: ICLR 2025 Poster (2025)
Keywords: Cross-modal representation learning
Molecule-text understanding
Alignment
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose a novel framework that improves drug design by effectively integrating molecular and textual information through cross-modal representation learning.

Abstract: Molecule-and-text cross-modal representation learning has emerged as a promising direction for enhancing the quality of molecular representation, thereby improving performance in various scientific fields. However, most approaches employ a global alignment approach to learn the knowledge from different modalities that may fail to capture fine-grained information, such as molecule-and-text fragments and stereoisomeric nuances, which is crucial for downstream tasks. Furthermore, it is incapable of modeling such information using a similar global alignment strategy due to the lack of annotations about the fine-grained fragments in the existing dataset.
In this paper, we propose Atomas, a hierarchical molecular representation learning framework that jointly learns representations from SMILES strings and text. We design a Hierarchical Adaptive Alignment model to automatically learn the fine-grained fragment correspondence between two modalities and align these representations at three semantic levels. 
Atomas's end-to-end training framework supports understanding and generating molecules, enabling a wider range of downstream tasks. Atomas achieves superior performance across 12 tasks on 11 datasets, outperforming 11 baseline models thus highlighting the effectiveness and versatility of our method. Scaling experiments further demonstrate Atomas’s robustness and scalability. Moreover, visualization and qualitative analysis, validated by human experts, confirm the chemical relevance of our approach. Codes are released on ~\url{https://github.com/yikunpku/Atomas}.

Published as a conference paper at ICLR 2025
ATOMAS: HIERARCHICAL ADAPTIVE ALIGNMENT ON MOLECULE-TEXT FOR UNIFIED MOLECULE UNDER- STANDING AND GENERATION
Yikun Zhang" Geyan Geyan
Chaohao Yuan', Bo Han Long-Kai Huang', Jianhua Yao',
Wei Liu', Yu Rong i
Tencent Al Lab *Peking University $DAMO Academy, Alibaba Group TTsnnguua University "Hong Kong Baptist University
(yikun. zh, yu. yung)n)) @hotmail com, blazeryy@tencent..com
ABSTRACT
Molecule-and-texx cross-modal representation learning has emerged as promising direction for enhancing the quality of molecular representation, thereby improving performance in various scientific fields. However, most approaches employ global alignment approach to learn the knowledge from different modalities that may fail to capture fine-grained information, such as molecule-and-tex fragments and stereoisomeric nuances, which is crucial for downstream tasks. Furthermore, it is incapable of modeling such information using a similar global alignment strategy due to the lack of annotations about the fine-grained fragments in the existing dataset. In this paper, we propose Atomas, a hierarchical molecular representation learning framework that jointly learns representations from SMILES strings and text. We design a Hierarchical Adaptive Alignment model to automatically learn the fine-grained fragment correspondence between two modalities and align these representations at three semantic levels. Atomas's ' end-to-end to- training framework supports understanding and generating molecules, enabling wider range of down- stream tasks. Atomas achieves superior performance across 12 tasks on I datasets, outperforming baseline models thus highlighting the effectiveness and versatility of our method. Scaling experiments further demonstrate Atomas's 's robustness and scalability. Moreover, visualization and qualitative analysis, validated by human experts, confirm the chemical relevance of our approach. Codes are released on wttpw//wwww..c....././e.
INTRODUCTION
Molecular representation learning is crucial in fields like drug discovery (Drews) 2000 Liu al.
2023b), virtual screeming (Walters et al. 1998 Goel et al. (023), and molecular design Yee et al. 22023
Thomas al. 2023). Recent advances in molecule-and-texx cross-modal models (Liu et al. 2023c
Luo et al. 2023 Liu et al. 2023e) have enhanced the generalization of molecular representations
by integrating internal structures (SMILES strings, structural data) and external domain knowledge (textual descriptions, knowledge graphs).
However, current approaches encounter three primary challenges. (1) Fine-Grained Correspon-
dence: Existing molecule-and-tex alignment methods (Christofidellis et al. 2023 Edwards et al.)
2022 Liu et al. 2023c Luo et al. [0023] Liu et al., 2023e) struggle to effectively capture fine- tine-grained correspondence related to local parts within different modalities, which is essential for downstream molecular tasks (Xia et al. 2023). For example, molecule captions generated by global alignment often fail to distinguish between 'D-glutamate' and 'L--glutamate' enantiomers, indicating lack of sensitivity to subtle details. This oversight can lead to inaccuracies in chemical analysis and interpretation. Currently, there is lack of fine-grained datasets with explicit annotations of local correspondences between molecules and text. Acquiring such datasets is difficult due to the com-
"Equal contributions.
Yu Rong and Geyan Ye the corresponding authors.

---

Published as a conference paper at ICLR 2025
NICENICCKKICP>> C2 CCC CAY O,
Molecule Retrieval
Retrieval
Testsn-Molecsi Retricval
Cycleropyl carboxylic flooro pipenorin 1-yl Bloro
Ato mas
Geneeaatonn
Melecula Generation
Ciprodlonacin qumnlin-4([B)) [Ry-one beaning syclopropyl, earootylic soid. Junm and substinents
7, respectively.
Caption
Figure 1: Atomas is a hicrarchical, end-to-end model designed to discover and automatically align local substructures of input while performing conditional generation. The learned cross-modal representations can be adapted to both understanding tasks (retrieval tasks) and generation tasks.
plexity and specialization of molecular strings and text descriptions, which require extensive human expert annotation. This limitation hinders the ability of globally aligned methods to effectively learn fine-grained information and address fine-grained alignment challenges. (2) Molecular Modality Focus: Current fine-grained alignment methods (Feng et al.) 2023 Ji et al. 2022) focus on molecular modality and fine-grained aligning substructures. The cross moddaltyy alignment molecular fragments and textual descriptions is largely overlooked. Existing segmentation tools for text and SMILES struggle with complexity and specialization, making it challenging to construct hierarchical text- molecular pairs. (3) Generative Task Optimization: Most approaches (Feng et al., 2023 Yu et al..
2024 Ji et al. 2022 Liu et al.) 2022) are designed for prediction tasks and do not optimize aligned
representations for generative tasks.
To this end, we propose Atomas, hierarchical cross-modal molecular representation learning framework that jointly learns representations from SMILES and text. Figure provides a conceptual illustration of Atomas. In Atomas, we exploit the unique characteristic of SMILES specialized form of text and employ unified encoder for both SMILES and text modalities. This results in more isomorphic representations for both modalities, thereby facilitating subsequent alignment tasks. Meanwhile, considering the textual description of molecules naturally has a hierarchical structure and the need for local molecular alignment, we design a Hierarchical Adaptive Alignment (HAA) model which comprises two components: Adaptive Polymerization Module (APM) and Weighted Alignment Module (WAM). APM assigns the quantities of the low-level tokens into high-level tokens (fragments) for the single modality, while WAM leverages token representations from both SMILES and text modalities to automatically learn the matching of token and aligns the representation two modalities in a set-wise manner. By iteratively invoking APM and WAM, we devised a three-level alignment scheme (atom level, fragment level, and molecule level). This hierarchical alignment structure enables improved learning of local alignment across different abstraction levels within the two modalities. Additionally, by incorporating conditional decoder within the alignment process, Atomas can optimize the representation of molecule and text specifically for generation tasks. Our contributions are summarized as follows:
To the best of our knowledge, Atomas is the pioneering molccule-ande -nd-ttee--nd-extt representation learning framework that tackles the challenge of aligning local information without the need for explicit labeling between text fragments and molecular substructures.
We introduce the concept of Hicrarchical Adaptive Alignment, enabling automatic learning the fine-grained correspondence between molecule and text three semantic levels from coarse fine. Atomas achieves state-of-the-ar performance on a wide range of molecule-text tasks, including molecule and text retrieval, text-based de novo molecule generation, and molecule captioning.
Atomas brings new insights into molecule generation tasks: (1) Aligning before generation improves the efficacy of molecule conditional generation tasks. (2) Fine- grained hierarchical alignment enhances the quality of controllable molecule generation. (3) Joint optimization within unified training framework surpasses the efficacy of a two-stage approach for molecular generation tasks. (4) Employing unified encoder may advantageous in scenarios characterized by data scarcity.
2 RELATED WORKS
Molecule-and Text Cross-Modal Models: (Edwards et al. 2022 Christofidellis et al. 2023 Lit
et al. 2023f Luo et al. 2023 Chen et al. 2024) investigate the interaction between molecular
2

---

Published as a conference paper at ICLR 2025
SMILES CI-C--Q--CIX--00
Molecule Caption
Global Allgunent
medecaate carpprred comprridag beneasa ring cart carrying aa/booyllc sedd vub rtitoer..
Unified
Moneacum Madel
componnd
Enesder
competsing henzene carriing carbosylic
Honarchical Adaptive
CL-CO-CT--CDC--00 Molecule Generation
Algament
Description
Adspthee Alignneet
Adaptive merizatinnn Midule Assgrewet Siop"
Weighted Alignneett Modalc
Stepl: Alkganer Shp
Supll: Merg Sty
Figure 2: Illustration of the proposed Atomas. Atomas is composed of four components. (I) Unified Encoder encodes both the input molecule and corresponding textual description. (2) Global Alignment module projects and aligns the global features of the molecule and text. A momentum model is used to ensure alignment consistency. (3) Hierarchical Adaptive Alignment aligns the molecule and text at three levels, including the Adaptive Polymerization module which clusters the original token features into distinct representation sets, and the Weighted Alignment module which aligns modalities in set-wise manner. (4) Conditional Decoder takes the molecule and text embedding as input and generates the target modality.
structures and textual descriptions to complement and enhance the overall information content. However, these methods only consider global representations of SMILES and text, overlooking finer-grained modal interactions. Atomas introduces hierarchical fine-grained alignment between SMILES and text, which crucial for controlled molecule generation and molecular captioning, enabling better performance. While (Edwards et al. 2022; Christofidellis et al. 2023) treat conditional generation tasks as translation tasks without establishing alignment, Atomas demonstrates the efficacy of performing preliminary alignment. Unlike (Liu et al. 20231 Luo et al. 2023 Chen et al. 2024)'s 's two-stage training strategies, Atomas employs a joint optimization approach, which is essential for effectively learning and generating molecular representations. More discussions are provided in Appendix A.
(Feng et al. 2023; Yu et al. 2024) focus on fine-grained aligning molecular modalities and tailoring for prediction tasks. Different from these paradigms, Atomas addresses modsl learning between molecule and text modalities. Chal- lenges arise from the lack of expert fine-grainee textual annotations for molecules and difficulty in constructing positive and negative pairs, text fragment may suit multiple molecule substructures. These challenges make Atomas' achievements this field particularly noteworthy. See Appendix[A.. for more discussions.
Molecular Modalities Representation Learning:
METHODOLOGY
In this section, we present the details of each component of Atomas. Figure illustrates the overall framework of Atomas. The algorithm details are shown in Appendix D
3.1 MOLECULE-TENS UNIFIED ENCODING
For SMILES-text pair M (S,T) T). SMILES S and text description T' are fed into the unified encoder fo since both are essentially languages. We discuss the advantages of utilizing unified
encoder in Section 4.6 The input 7 with No. tokens is embedded into word sequence T, {1t No.
i=1

---

Published as a conference paper at ICLR 2025
where tg IR denotes the feature vector of the i-th word. The input S with Nee tokens is embedded No. into atom sequence S, = {0%) j=1' where &y E R Since the T5 model does not have the [CLS] token, we first aggregate the Ta and Sa by projection module proj (.) to obtain the global feature by and Sy;
lg proj((()) = W, To + b., 8g proj(S.) W S. + bb.
where W. E NiXl and W, E ER N.1 are the learned matrix, b, and b, are the bias terms.
We then align the global representation pair (lg. 8,) by performing cross-modal contrastive learning. To ensure sufficient negative pairs and consistent feature representation from both modalities, we follow (He al. 2020 introduce a momentum unified encoder fā and two queues for text and SMILES s denote as Q2 and Q,, respectively. fø is updated by fo in the following way,
fØ a f (1 ax)
where a E [0, 1) is a momentum coefficient parameter and only the parameters fo are updated by back noc-ppropagttion
Qi and Qs store the global feature ty and sg generated by fø" thereby creating two large and consistent dictionaries that cover a rich set of negative samples. By doing this, we calculate the global similarity score of -MILLE within the specified queue range instead of a
exp(sim(f,,- s///T)
(, D9 exp(sim((t) Q2)/T)
where is a learnable temperature parameter. and sim(::-) is the similarity metric, here calculate it using the cosine similarity function. Similarly, we can obtain the global SMILES-to-text similarity score (y,s t') )'). Inspired by (Li et al. 2021; 2022), soft labels are created from the momentum encoder f&"h as training targets to account for the potential positives in the negative pairs. Then the global alignment loss go can be formulated as:
Lya [[(1 (1118)y B)y ( )loggSS s s')) + [(1 (1- ")y (s (loggsS S,(ss t'))},
(4)
where is a hyperparameter controlling label smoothness. y 0² and y (2s denote ground-truth similarity, with negative pairs assigned a probability of 0 and positive pairs probability of 1.
3.2 HIERARCHICAL ADAPTIVE ALIGNMENT
Given an encoded SMILES-text pair M- = (S, it is challenging to explicitly extract the corre- sponding fine-grained information (e.g., functional groups in SMILES and phrases in text) from S and T. To address this, we propose an adaptive polymerization module that clusters token-wise features into disentangled representation sets. Subsequently. module to estimate the correlation between the two modalities and identify potential active units in set-wise manner. Figure 2 illustrates the framework of hierarchical adaptive alignment. The adaptive polymerization module includes an assignment step and a merge step. The weighted alignment module performs the alignment step. By stacking these two modules, we expand the fine-grained alignment between SMILES and text to the hierarchical interaction. The effectiveness of this hierarchical adaptive alignment is demonstrated Table 7
introduce weighted alignment
Specifically, we perform hierarchical adaptive alignment at three levels: atom level, where atom is aligned with word: fragment level, where functional group is aligned with phrase; and molecule level, where molecule is aligned with paragraph. Thus, it process alternates between three steps in level-wise manner: assignment step, merge step, and alignment step.
Assignment Step: We utilize learnable token aggregation module to implement adaptive poly- merization. The density peak-basc -ased clustering algorithm with k-nearest neighbors (Du et al. 2016) Yao et al. 2019 Zeng et al. 2022a Jin et al. 2023) (DPC-KNN) is utilized to assign tokens to Ne.
clusters. Starting with the atom level, we firstly take atom (word) token features S. = { { j=1
into
one-dimensional convolution to extract local features.
S. LayerNorm W, b) + S.).
4

---

Published as a conference paper at ICLR 2025
Then we compute the local density p of each atom token feature sy according to it's k-nearest neighbors:
1 sall2
Pj exp
1
Σ
+G,
(6)
k:
VD,
$ E KNN (s"))
where s and s are their corresponding SMILES token features. D, is the channel number of SMILES token features. KNN (s²)) denotes the k-nearest neighbors of an atom token j. is random noise that randomly sampled from the uniform distribution within the interval [0.1), ensuring that no tokens have the same density.
Then, we calculate the distance indicator for each token feature si by determining the minimum distance between it and any other token possessing higher local density. As for the token with the highest local density, its indicator is determined as the maximum distance between it and any other tokens:
minaap>p 2:p max, |s S
|1
if s... Pi Pj otherwise.
6,
Here, serves as an indicator of the local density of tokens, which reflects the number tokens located in the vicinity of sq. 5 represents the distance of token from other high-density tokens, which measures how far is from other tokens that are also located in highly dense regions. Together, P and 8 provide valuable information about the distribution and proximity of s S..
We identify tokens with relatively high values of p x as cluster centers and then assign all other tokens to their nearest cluster center based on the Euclidean distance d. This clustering approach enables us to decode the input tokens into coherent semantic units, providing a more structured and meaningful representation for both word sequence T, and atom sequence Sa.
Merge Step: Tokens with similar semantic meanings may not have equal importance, so in the merge step we first assign weight each token feature and calculate the weighted average token features of each cluster to represent the corresponding cluster:
S
S
(8)
where (L..(S.) is the weight of each token feature in S.. No represents the number of features within the j-th cluster at the fragment level, S is the k-th token feature of S, and wy the corresponding weight score. 5 is the j-th weighted average token feature.
Then we apply attention mechanism merged token features. Sm are used queries Q and key K, value V corresponding to the original token features S.. We take the resulting output of the
attention module as higher semantic level features, i.c., functional group sequence S 5 { 8
N.r
j=1
where 8 R Perform the same operation of the above steps word tokens to get phrase sequence Ty = { i=1' where f E R. Nef represents the number of clusters formed by word tokens. We repeat this process the fragment level to obtain the molecule-level features.
Nes "
Alignment Step: After the assignment and merge steps, tokens are polymerized into semantic units. we perform the weighted alignment module (Wang al., (022) on each level in set-wise between the SMILES and text to get the weighted average maximum alignment score. Starting with the atom level, we can obtain the -MILEE similarity score:
Nis
Shaa 8) Σ w (T.)
max Oejs aij
where w. = Softmax Coftmax(MLP... (T.)) represent the learnable weights for each textual token. The normalized alignment score asy captures the similarity between the i-th description token feature and j-th SMILES token feature. Then the hierarchical adaptive alignment loss at the atom level can be calculated as:
5

---

Published as a conference paper at ICLR 2025
1 hha 2
log exp( Saa (tx. 8k]/T)
1
B Σ k
log exp( han a (sk. tx)
+
(10)
Σ exp(Shoa (tg, 81) 81)
B
Σ Exp(Shad haa (8), tx) tx)
where B is the batch size and is the temperature hyperparameter. The same alignment operation is performed the fragment level and molecule level to get C) and L
3.3 CONDITIONAL GENERATION BASED ON ALIGNED REPRESENTATION
We employ a conditional generation approach to generate the target modality based on the aligned Ni
and S. { representations denoted T. = { [ :=1
[=]==1 Na In text-based molecule generation
task, the decoder takes an aligned textual description T. as input. The decoder then iteratively attends to previously generated tokens 35 via self-attention and input condition Ta via cross-attention. Using these attended representations, the decoder predicts the probability of future SMILES tokens P( (6 Ta)- Then the decoder be optimized by minimizing the negative log-likelihood of label SMILES tokens given textual description T, and the same operation is applied the molecule captioning task:
Non
Cim -
Σ
log lsg ,
(11)
3.4 TRAINING OBJECTIVES
The goal of Atomas is to align the molecule and text at different levels of granularity while condition- ally reconstructing the molecule or text description. We jointly optimize the global alignment loss Eyas hierarchical adaptive alignment loss Lrea and language modeling loss fim in end-to-end manner. The overall loss function of Atomas simultancously:
min Lga + Praa
Lims
Lhaa Lhea + L fheaa
(12)
where & denotes all learnable parameters of Atomas, Chaor Chaa- Chan operates at the atom level, fragment level, and molecule level, respectively.
4 EXPERIMENTS
In this section, we present the quantitative and qualitative results of Atomas. The experiment is set to evaluate the effectiveness of Atomas four aspects: (1) improving the efficiency of retrieval and property prediction tasks, (2) enhancing the generation capability of generation tasks, (3) evaluating the effectiveness each module, (4) chemical significance and scalability.
4.1 INITIAL TRAINING
Dataset and Training Details: We follow the MoleculeSTM's pipeline (Liu et al. 2023c) collect molecu- lar SMILES-text pairs from PubChem website. Pairs with the same Pub- Chem ID and descriptions shorter than 18 characters are merged, and dupli- cates are removed from the down- stream task datasets to prevent data leakage. This process results in a high- quality dataset of 51,340 unique pairs, which is used for the initial training phase. Dataset details and statistics provided in Appendix E The model is trained 8 NVIDIA Tesla A100-SXM4 GPUs
Table 1: Performance comparison molecule-text re- trieval task. Bold and underlined indicate the best and second-bee best results, respectively. Details are provided in Section 4.2 and Appendix
(No Fine-tuning)
RØ5
Molecule
RØ10
MRR
R@1
RØS
MRR
IDISMIFFS a 279 Gropli
MoMu
4,50
14,48
20.99
10.33
3.08
1232
18.95
9,89
MalCA
35.09
62.14
69.77
47.33
27.95
66.11
74.48
50,80
JD SMILEY
Anowledge
MoleculeSIM
25.00
39.50
Atomass buse (Ours)
39.08
39.72
66.56
47.33
27.88
59.22
65.56
47.81
Atomas-large (Ours)
4908
68.32
73.16
$7.79
46.22
66.02
72.32
55.52
using a batch size of 16 SMILES -tett pairs, no weight decay, and the learning rate is set to 1e-4. More implementation details provided in Appendix F

---

Published as a conference paper at ICLR 2025
Table 2: Performance comparison on text-based de novo molecule generation. Bold and underlined indicate the best and second-best results, respectively. "I" denotes that higher is better. " denotes that lower is better. We repeat the Atomas-large 3 times and report the average with 95% confidence interval. Details are provided Section 4.3 and Appendix G
Model
BLEUT
Exactt
Levenshtein
MACCS FIST
RDK FTS
FTS
Validity?
TI SMILES 212 Graph 2/2 Imoge
GiT-Mol
0,756
0.051
26.32
0.738
0.582
0.519
0.928
Gruph Knowledge Gruph
MolFM small
0,803
0.169
20.868
0.834
0.721
0.662
0.859
MolFM--asc
0.822
0.210
19.45
0.854
0.758
0.758
0.892
2D Graph
ICMAIGalacties 225
0.836
21.480
0.893
0.809
0.743
0,825
ICMA(Misra--PBB.
10 SMILES
MoflS-small
0.749
0,082
28.816
0.780
0.654
0.601
0.725
MolIS-base
0.779
0.082
25.19
0.788
0,662
0.602
0.787
Moolrr-large
0,854
0.318
16.32
0.889
0.813
0.750
0.958
T5-augm
0.322
0.816
MolXPT
0.215
0.859
0.757
0.667
0.983
MotReGPT -.5tuu.-t
0,790
0.139
24.91
0.847
0.708
0.624
0.887
(GPT-4-0413)
0.280
Atomas-buse (Ours)
0.868
0.343
13.76
0,908
0.827
0.773
0.971
Atomas-large (Ours)
0.874
0.387 .006
0.914
0.841
+-002
0.788
4.788
-003
Table 3: Performance comparison on molecule captioning task. Bold and underlined indicate the best and second-best results, respectively. We repeat the Atomas 3 times and report the average with
a 95% confidence interval. Details are provided in Section 4.4 and Appendix G
Model
#Params
BLEU-2
BLEU-
ROUGE-1
ROUGE-2
ROUGE-L
SMILES Gruph
MoMu-small
82M
0.532
0.445
0.564
McMu-base
252M
0.549
0.462
0.575
MoMu-large
782M
0.599
0,515
0.593
InstructMol GS
6.9B
0.475
0.371
0.566
0.394
0,502
MoICA. Galac1.3B
1.3B
0.620
0,531
0.681
0.537
0.618
SMILES
Graph Image
GFT-Mol-GG8
700M
0,352
0,263
0.575
0.485
0,560
SMILES
Graph + Knowledge Graph
MolFM small
136.2M
0.542
0.452
0.623
0.469
0.562
MolFM-base
296.2M
0.585
0.498
0.653
0.508
0,594
SMILES
MoITS-small
77M
0.519
0.436
0.620
0.469
0.563
MolTS-base
248M
0,540
0.457
0.634
0.485
0.578
MolIS-large
783M
0.594
0,508
0.654
0.510
0,594
Text+Chem T5-augm
220M
0.625
0,542
0.682
0.543
0.622
MolXPT
350M
0.594
0,505
0.660
0.511
0,597
MolReGPT (GPT-3.5-tubo)
>175B
0.565
0.482
0.450
0.543
0.585
MolReGPT (GPT-4-0314)
0.607
0.525
0.634
0.476
0,562
Atomas-base w/o Initial training (Ours)
271M
0.6045 0000
0.5185 0001
0.6745
0.5315
0.6155
Atomas-base (Ours)
271M
0.6322.00x
0.549-t-00z
0.685=t.003
0.5-55=-001
0.626
4.2
MOLECULE-TER RETRIEVAL
Set Up: To evaluate Atomas's performance and generalization, we use the PCdes dataset (Zeng et al. 2022b instead PubChem dataset, which includes 15,000 molecule pairs. Following MolFM (Luo al. 2023). apply scaffold splitting to divide the dataset into training, validation, and test sets a 7:1:2 ratio. We directly evaluate Atomas and other baseline models on test sets without fine-tuning. For inference, retrieve results directly from the entire test dataset without first selecting top-k candidate set. We assess performance using Mean Reciprocal Rank (MRR) and Recall at 1, 5, and 10.
demonstrates that Atomas outperforms recent state-of-the-ar methods in both text-to-molecule and molecule-to-tex retrieval tasks on R@1 1. This indicates that multi-level fine- grained interaction and alignment can yield significantly better outcomes than methods based only on coarse-gree grasind representations. For detailed introduction to the baselines, refer to Appendix G.2
Results: Table I
4.3 TEXT-BASED DE Novo MOLECULE GENERATION
Set Up: ChEBI-20 (Edwards et al. 2022 gold standard dataset extensively used for molec- ular generation tasks. It comprises 33,010 poolecule-descrpptie pairs and is split into 80/10/100 10/10% train/ mrin/valiiation/tes sets. To assess our model's performance, we use standard metrics for the genera- tion task. More details are provided in Appendices and G.1

---

Published as a conference paper at ICLR 2025
Table 4: Performance comparison on molecule property prediction. We present the ROC-AUC (%) scores molecular property prediction task on MoleculeNet. We use scaffold split following MoleculeSTM. We repeat the Atomas 3 times and report the average with 95% confidence interval.
Method
BBBP
Tox21
ToxCust
Säder
Clin
MUY
HIV
Bace
Avg
MileculSTM
70.75E1990
15.7180.89
65.1720:37
No. 66.60022.28
63.6911.46
770220044
81.9520.41
73.33
MolFM
77.2+0.7
64.2+0.9
76.0088
78.8+11
83.9=111
McMu
70.5+2.0
75.640.3
63.4+0.5
60.5209
79.9.4,1
70.5114
75,9:0.8
76.722.1
71.63
MoICA-SMILES
70.840.6
76.040.5
56,240.7
61.121.2
89.0&1.7
79.320.8
72.1
Alomas
73.7261.67
77,8840.36
66,9440.9
64.40:1.9
93.1620.5
76,30
80.55:0.43
83.14-1.1
77.01
Table 5: The scaling of the initial train- ing dataset on molecule generation task. Atomas consistently surpasses baselines in limited data size.
Table 6: The scaling of the model size on
molecule generation task. Increasing Atomas
parameters enhance generation performance. Complete indicators are in the Appendix H.T
Model
Data sizes
Fxaet"
Levenshtein,
RDK FIS!
Model
Modil
Levesstrin
RDK
MallMM
0.755
MaReeP (4-1133)) 51758
MolTS-large
Atomas buse
15k
0.318
14.68
0.817
271M
Moomas-aase
51k
0.343
13.76
0.827
Atemas-large
825M
12.70
Quantitative Results: Tables 2 and 14 show the text-based de novo molecule generation per- formance. Atomas outperforms all baseline models in all metrics. We also calculate the scores of molecule novelty to show that Atomas can perform well generation. Since SMILES is the dominant molecular representation, Atomas uses SMILES ES and compares only with methods based on SMILES. Methods like (Liu et al. 2023a Luo et al.) 2023 use separate unimodal pre-traine pre-trained models for different modalities, complicating information exchange and limiting fine-graine, interactions. Their two-stage training process also restricts generative capabilities. Additionally, GPT-like and mecoerr-dccoder-basee methods (Edwards et al. 2022 Christofidellis et al. 2023 Liu et al. 2023d
Li al. 2023a) miss the benefits of well-aligned multimodal representations.
4.4
MOLECULE CAPTIONING
Set Up: We evaluate Atomas for molecule generation using the ChEBI-20 dataset. Evaluation metrics include BLEU-2, BLEU-4, ROUGE-I, ROUGE-2, and ROUGE-L. We present more details in Appendices and G.T
Quantitative Results: Table presents the overall molecule captioning performance. Atomas surpasses all baseline methods across all evaluation metrics. Notably, our Atomas-base model outperforms the MolTS-large model while using only 35.0% of its parameters and requiring no initial training. highlighting the effectiveness of our proposed framework.
4.5 MOLECULAR PROPERTY PREDICTION
We evaluate Atomas eight binary classification datasets from MoleculeNet. We use scaffold split following (Liu et al. 2023c The evaluation metric the area under the receiver operating characteristic curve (ROC-AUC) Table shows that we have consistent improvements on seven out of eight tasks compared to the baseline models using the SMILES string as input. The overall performance exceeds that of all baseline models.
Set Up and Results:
4.6 ABLATION STUDY
The Scalability and Robustness of Atomas: Table 5andd outperforms baseline methods in both the scaling of training dataset and the scaling of model size. We also explore how Atomas's performance varies with complex molecular structures and textual descriptions. In the ChEBI-20 test dataset, molecules categorized into length intervals of 100, with "Mol_len 100" representing lengths between 100 and 200. Similarly, the input molecule descriptions categorized by length, with "Text len 100" indicating descriptions between 100 and 200 characters. As shown in Tables [5 and 16] Atomas performs more robust performance than baseline methods on complex molecular structures and highly technical textual descriptions.
that Atomas consistently
Unified Encoder Better than Separate Encoders: To investigate the impact of using a unified encoder versus separate encoders for text and SMILES, sample 75%, 50%, and 25% of
8

---

Published as a conference paper at ICLR 2025
Table 7: Ablation study for the effectiveness and time consumption of components on molecule generation task. The first row is the baseline without alignment to generate SMILES based on text. The second row is the baseline only using global alignment. More details are the Appendix H.3
Global Aligummet
Hierarchical Aligument
Conditional Generation
Exactf
Levencheein]
Morgun FTS?
Training Time(s/samlle
0.082
24,846
0.602
0.0112
0.223
16,946
0.716
0.0119
0.0132
0.298
15,472
0.750
0.0145
Table Ablation study for the effectiveness of joint optimization molecule retrieval task.
Table 9: The scores of molecule novelty
on text-based de novo molecule generation task. And The human expert evaluation on molecule caption task.The numbers in brackets indicate the number of ranks 1, 2, and 3. The lower rank score indicates better performance of molecule caption.
Text to Molecule
Training Strategy
R 0
R@5
R@10
MRR
2Stages
37.74
58.01
65.02
47.20
Joint optimization
39.08
59.72
66.56
48.47
Training Strategy
Molecule Text
Method
Noveltyl
Average Ranking of Human Expert Evalnation
R@1
R@5
R@10
MRR
2Stages 36.54
57.31
63.58
46.10
TexeeChem 15umgm
MolTS- 0.76
0.34
2.6(0/233)
2.2(172225
Joint optimization
37.88
59.22
65.56
47.81
Atomas-large
0.85
1,2/4/1/0)
the training set as training subsets, based on the distribution of text lengths, and evaluate on the original validation and test sets. As shown in Figure 3 the performance of the separate encoders SSep-Encoder significantly declines as the training sets decrease, compared to the unified encoder (Uni-Encoder). These findings may provide insight into molecular design, where data scarcity is common challenge. Complete performance details are provided in Appendix H.2
The Effectiveness of Hierarchical Alignment:
The scarcity of fine grainer molecule and text pair datasets makes it challenging to quantify the model's ability to capture fine-grained information, unlike the vision community, where extensive datasets exist to evaluate models fine-grained image recognition tasks. Nonetheless, we can implicitly validate the effectiveness of Atomas by observing improvements in component ablation studies. Table 7 provides clear insights into the effectiveness of fine-grained alignment. Figure 4 (right) shows the effect of adaptive alignment level numbers. We find that the model achieves the best performance at 3-level numbers.
The Computational Efficiency: Table 7 reports the average training time on the ChEBI-20 dataset using an NVIDIA A100 40 GB GPU and the performance on the molecule generation task. While hierarchical alignment slightly increases training time by just 0.0026 seconds, it significantly boosts Atomas's 's overall performance. The increase inference time is almost negligible. This result highlights the effectiveness of our efficient design.
Joint Optimization Benefits Both Learned Representation Quality and Generation Task Per- formance: T and Figure 4 (left) show the ablation study on molecule retrieval and generation task for the different training strategies. The "Baseline" refers to the MolT5-base model. From these results, we can conclude that the essence of joint optimization is the mutual facilitation between the caption/generation task and molecular representation learning (retrieval tasks) (Bahdanau et al. 2015). More detailed discussions are provided in Appendix H.6
Merging Methods Comparison: We compared Atomas with other molecular decomposition meth- ods like BRICS decomposition method. The decomposition by BRICS resulted in fragments like" [ which introduces additional characters and does not allow for hierarchical decomposition, limiting its direct replacement of the Adaptive Polymerization Module in Atomas.
4.7 VISUAI.ZZATION AND QUALITATIVE ANALYSIS
Visualization: To better understand Atomas, we present the visualization of the adaptive polymeriza- tion module in Figure 5 The molecule formed by combining atoms at positions 0-15 and at sites 16-26 through dehydration condensation. From atom level to fragment level, Atomas clusters atoms (words) into functional groups (phrases) using an adaptive polymerization module. From fragment level to molecule level, Atomas clusters atoms at sites 0-13 and 15 together to form monomer-like structure. This indicates Atomas tends to focus on macro-level information as it ascends the hierarchy.

---

Published as a conference paper at ICLR 2025
25ggo Jeistl Opteniottonn
Basel]
Margan FTS
Levcnllicte
Uni-Enewder
Sep-Facader
0,450
0.225
00130
6:000
6:90
6.715
8.755
6.15
6,750
0
0,09
197%
50%
25%
10075 75%
30%
25%
100%
50%
10%
0.21
Figure 3: Unified encoder vs separate en- coder with the scaling dataset. Evaluate molecule generation task.
Figure 4: Ablation study for the effective- ness of joint optimization (left) and hierar- chical alignment level numbers (right).
Abtea
Frogment
Diagram
as glacosyl hydrocyciananii and that in the rinromote onter obtsined by the carbury group
goop
molecule glatosy hydooeycinnntic
simpie meseent hyyhosy hoteie Do
purbary group tess--rrope Einsamane
and thet the cirrerritr ourr ohaissdd by the mngg
alassse moloasle glazoey!
modecule formad condmaationn
manositapic and with the stcmoric bydhssy grop bes--aaaa
hyshasydrrumme and thet the cinveroon formed
glanegyl hyyeroyrinraaii innamic scidl the onter
mamemmppannn with the snomone tyshmoy group
with group
Figure 5: The visualization of adaptive polymerization module. The process of atom (word) polymerization to form individual sets is illustrated at three levels, including the reference diagram, from left to right. Atoms (words) belonging to the same set are highlighted using the same color.
Matec logal
Melicade Crypine
Groed Trok
Alimms Ohiget
panue Quaen's di yaing glo
The cocooaaiiaaaao Clapplic
Applide by
LCysi ond
mabe de de Li-piie
Figure 6: Comparing the performance of global alignment methods with Atomas molecule captioning task.
Figure 7: Comparing the performance of global alignment methods with Atomas on molecule generation task.
Qualitative Analysis of Molecule Caption: shown Figure 6 global alignment methods like (Christofidellis et al.) 2023 Edwards et al. 2022 Pei et al. 2023) struggle to distinguish between "D-glutamate" and LLLglutamate enantiomers. In contrast, Atomas generates more accurate and detailed molecule descriptions, demonstrating the effectiveness of hierarchical alignment models.
Qualitative A Analysis of Molecule Generation: As shown Figure 7 gloabl alignment methods like (Christofidellis et al. 2023 Edwards et al. 2022 generate the "AMP" structure but miss fine-grained details like "2- ">-hyrrox"" Conversely. Atomas successfully generates the right structure.
To provide additional validation of the model's performance and practical utility, we incorporated human expert evaluations. We randomly selected five molecular captions of varying lengths generated by Atomas, Text+Chem T5, and MolT5. The average rankings from these evaluations are shown in Table Atomas achieved the overall best performance among the three models, ranking first in 4 out of generated molecular captions. The corresponding SMILES samples are provided in Appendix !
Human Evaluation:
5 CONCLUSION
We introduce Atomas, a hierarchical alignment framework designed to enhance molecular representa- tion by aligning SMILES strings with textual descriptions through Hierarchical Adaptive Alignment model. Atomas excels at capturing fine-grained details, achieving state-of ffute-of-the-ee of-the-artin retrieval, property prediction, and molecule generation tasks while demonstrating robust scalability and generalizability.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
Yikun Zhang would like to express sincere gratitude to everyone who contributed to this work, providing support and guidance throughout the process. Their valuable insights and assistance were instrumental in bringing this research to completion and enabling its publication at ICLR.
ETHICS STATEMENT
This paper does not involve crowdsourcing or research with human subjects. The proposed method poses no high risk for misuse. We cite the original paper that produced the code package or dataset. Below is the broader impact of our research:
For machine learning community: this study, we introduce a Hierarchical Adaptive Alignment model for automatically learning fine-grained information. This offers novel approach to facilitate the lineggriieed learning of extensive unlabeled datasets in diverse domains.
For the drug discovery community: Atomas utilizes the uni-encoder to alleviate the problem of limited data in the specific domain of molecular studies. This approach offers a novel training methodology for the data-scarce molecular drug discovery domain. Atomas's end-to-end training method involves alignment followed by generation and demonstrates superior performance in molecular understanding and generation tasks. Furthermore, Atoms provides the way for high-precision controllable molecular generation research. The adap- tive alignment module Atomas presents an efficient solution for leveraging large-scale unlabeled biochemical texts. We hope that the tross-modal representations learned from Atomas can be applied to variety of molecular downstream tasks such as virtual screening and molecular drug design.

---

Published as a conference paper at ICLR 2025
REFERENCES
Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, 2015.
He Cao, Zijing Liu, Xingyu Lu, Yuan Yao, and Yu Li. Instructmol: Multi-modal integration for build- ing versatile and reliable molecular assistant in drug discovery. arXiv preprint arXiv:2311.16200 2023.
Zhuo Chen, Yichi Zhang, Yin Fang, Yuxia Geng, Lingbing Guo, Xiang Chen, Qian Li, Wen Zhang, Jiaoyan Chen, Yushan Zhu, et Knowledge graphs meet multi-modal learning: comprehensive survey, arXiv preprint arXiv:2402.0539/ 2024.
Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, and Matteo Manica. Unifying molecular and textual representations via multi-task language modelling. In International Conference on Machine Learning, pp. 6140-6157. PMLR, 2023.
Jurgen Drews. Drug discovery: historical perspective. science, 287(5460):1900-185 1964, 2000.
Mingjing Du, Shifei Ding, and Hongjie Jia. Study on density peaks clustering based on k-nearest
neighbors and principal component analysis. Knowledge- Based Systems, 99:135-144 145, 2016. ISSN 0950-7051. doi: http://too.or/11000116j.....mmmm
URL https:/ www 6000 94
sciencedirect com/ ccom/scienceii
Joseph L. Durant, Burton A. Leland, Douglas R. Henry, and James G. Nourse. Reoptimization of mdl keys for use in drug discovery. Journal of Chemical Information and Computer Sci- ences, 127331227) 2002. doi: 10.1021/c1010132r httpttss::/ ptattps::/ddi.oor/11000 1021/ PMID: 12444722.
ci 11011132r
Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing, pp. 313--41,,,, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. 10153/861/1/// 2022. www.eemmwmmmm22222mm mmin.
www.022.mmllpmmii.... URL https: http://claannhhoocoooo org
Shikun Feng, Lixin Yang. Weiying Ma, and Yanyan Lan. Unimap: Universal amiles-graph represen- tation learning. arXiv preprint arXiv:2310.14216, 2023.
Manan Goel, Rishal Aggarwal, Bhuvanesh Sridharan, Pradeep Kumar Pal, and U Deva Priyakumar. Efficient and enhanced sampling of drug-like chemical space for virtual screening and molecular design using modern machine learning methods. Wiley Interdisciplinary Reviews: Computational Molecular Science, 13(2) 2023.
Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1729-97733 2020.
Zewei Ji, Runhan Shi, Jiarui Lu, Fang Li, and Yang Yang. Relmole: Molecular representation learning based on two-level graph similarities. Journal of Chemical Information and Modeling, 62 2022. doi: 10. 10.1021/acs.jeee jes.jecim.2c0079.. URL http::::://oiooooo/ooo/
acs. 2c0072
PMID: 36302249.
Peng Jin, Jinfa Huang, Pengfei Xiong, Shangxuan Tian, Chang Liu, Xiangyang Ji, Li Yuan, and Jie Chen. Video-text as game players: Hierarchical banzhaf interaction for cross-min moodl representation learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2472-2482, 2023.
Jiatong Li, Yunqing Liu, Wenqi Fan, Xiao Yong Wei, Hui Liu, Jiliang Tang, and Qing Li. Empowering molecule discovery for molecule-caption translation with large language models: A chatgpt perspective. arXiv preprint arXiv:2306.066615 2023a.
12

---

Published as a conference paper at ICLR 2025
Jiatong Li, Wei Liu, Zhihao Ding, Wenqi Fan, Yuqiang Li, and Qing Li. Large language models in-context molecule learners. arXiv preprint arXiv:2403.04197 2024a.
Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven
Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694-9705, 2021.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. In International Conference on Machine Learning, pp. 12888-12900. PMLR, 2022.
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping Einuage-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19742, PMLR, 2023b.
Sihang Li, Zhiyuan Liu, Yanchen Luo, Xiang Wang, Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, and Qi Tian. Towards 3d molecule-tent text interpretation in language models. In The Twelfth International Conference on Learning Representations, 2024b. URL BtKL/tttts://ppereeeet net bet/fforuu furmm witrrrmmiiiide did=x 4yN1 kach
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74 81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL
-ttp://hoo..oo..oo/0//0000000000 w04- 104131
Pengfei Liu, Yiming Ren, and Zhixiang Ren. Git-mol: A multi-modal large language model for molecular science with graph, image, and text. arXiv preprint arXiv:2308.06% 1. 2023a.
Pengfei Liu, Yiming Ren, Jun Tao, and Zhixiang Ren. Git-mol: A multi-modal large language model for molecular science with graph, image, and text. Computers in Biology and Medicine, 171:108073, 2024. ISSN 0010-4825. doi:
108000. 108073. URL https
www. www.ssieieeddrerct... com wtoo:/scinnce//ee http:///ciiinnaaaa....aap///////
Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pre- training molecular graph representation with 3d geometry. In International Conference Learning Representations, 2022. URL http:ttts:///peeeeeetee
wtt.///wwwew/f.rrrrr//// net/ forum? 2ttp://forummmddueeeeeeee pOKP am
Shengchao Liu, Hongyu Guo, and Jian Tang. Molecular geometry pretraining with SE(3)-invariant de- noising distance matching. In The Eleventh International Conference on Learning Representations, 2023b. URL http:httt///ppeeeeee
Shengchao Liu, Weili Nie, Chengpeng Wang. Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar. Multi-modal molecule structure- text model for text-based retrieval and editing. Nature Machine Intelligence,
2023c.
Zequn Liu, Wei Zhang, Yingee Xia, Lijun Wu, Shufang Xie, Tao Qin, Ming Zhang, and Tic- Yan Liu. MolXPT: Wrapping molecules with text for generative pre-training. In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki (eds.), Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 1606-1616, Toronto, Canada, July 2023d. Association for Computational Linguistics. doi: 10. 1/253//v12003.aallssshh https: https://aclanthin http:////aaaatthhoo...oooo org/ 202.0002322220... acl.
URL
Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. MolCA: Molecular graph-language modeling with cross-modal projector and uni-modal adapter. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 15623-15638, Singapore, December 2023e, Association for Computational Linguistics. doi: 18.18653/v 1/2023. 12022//.mm2mmm..mmann 966. URL http::::///aaaatahlooooo hwwp::olllo...yo//////o wwt.2222....mmmm..mmm emnlp-mainn 956
Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Molea: Molecular graph-language modeling with cross-modal projector and uni-modal adapter. arXiv preprint arXiv:2310.12790 2023f.
Yizhen Luo, Kai Yang, Massimo Hong, Xingyi Liu, and Zaiqing Nie. Molfi:: A multimodal molecular foundation model. arXiv preprint arXivv2307.0948. 2023.
13

---

Published as a conference paper at ICLR 2025
Hehuan Ma, Yatao Bian, Yu Rong, Wenbing Huang, Tingyang Xu, Weiyang Xie, Geyan Ye, and Junzhou Huang. Cross-dependent graph neural networks for molecular property prediction. Bioinformatics, 38(7):2003-2009, 2022.
Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to Information Retrieval. Cambridge University Press, 2008.
Frederic P. Miller, Agnes Vandome, and John McBrewster. Levenshtein Distance: Information theory, Computer science. String (computer science), String metric, DamerauuLevensiieee distance, Spell checker, Hamming distance. Alpha Press, 2009. ISBN 61 6130216904.
Janghoon Ock, Joseph Montoya, Daniel Schweigert, Linda Hung, Santosh K Suram, and Weike Ye. Unimat: Unifying materials embeddings through multi-modal learning. arXiv preprini arXiv:2411.08664, 2024.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin (eds.), Proceedings of the 40th Annual Meeting of the Association for Compulational Linguistics, pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics. doi: 10.3311/1073003310731111
URL https: http:///aaaaaatt.....eoa.aaa wwwporg//00-..... P22 1040
Qizhi Pei, Wei Zhang. Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, and Rui Yan. BioT5: Enriching mrosl integration in biology with chemical knowledge and natural language associations. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 1102-1123, Singapore, December 2023, Association for Computational Linguistics. doi: 10. 18.15v/3/v 201553/v1/2222/ 1/2023. emnlp-main.700 70. URL https: http:///aaaaatttooo..coo.. wwtp:/llooooo.///2220/ htt.://....rrr22233..mmmmmmmmmaa emnlp-main. 70
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 8748 8763. PMLR, 18 18-24 Jul 202 URL html
https::/proceediing htt.a/eeeaaaaaarrprrrrree/////// pttpp//ee//eeaafooeeeeoeeee/
mlr wwwps/eesseeeeeeeeeeeeeee
Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763 PmLR, 2021b.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 21 (140))1-67, 2020.
Alex Rodriguez and Alessandro Laio. Clustering by fast search and find of density peaks. science, 344(6191):1492-4499 2014.
David Rogers and Mathew Hahn. ditended-connectivee fingerprints. Journal of Chemical Information and Modeling, 5045):742-754, 2010. doi: 10.1021/ci100050. URL https
d/doi. wttgg/.../////////.. 1021 0021/111000 PMID: 20426451.
Nadine Schneider, Roger A. Sayle, and Gregory A. Landrum. Get your atoms in order-an open- source implementation of a novel and robust molecular canonicalization algorithm. Journal of Chemical Information and Modeling, 55(10):1111-2120, 2015. doi: 10. hiccs.cim.5b00543 PMID: 26441310.
URL https: httpp:::/doo. /oii org
1 021 / acs. jcim. 5b00543
Teague Sterling and John J Irwin. Zine 15-ligand discovery for everyone. Journal of chemical information and modeling, 55(11):2324-2333 2015.
Bing Su, Dazhao Du, Zhao Yang, Yujie Zhou, Jiangmeng Li, Anyi Rao, Hao Sun, Zhiwu Lu, and Ji-Rong Wen. A molecular multimodal foundation model associating molecule graphs with natural language. arXiv preprint wrXiv:2209.0588,, 2022.
14

---

Published as a conference paper at ICLR 2025
Morgan Thomas, Andreas Bender, and Chris de Graaf. Integrating puucture-basede approaches in
generative molecular design. Current Opinion in Structural Biology, 79:102559, 2023.
Ellen Voorhees. The trec-8 question answering track report. 11 2000.
W Patrick Walters, Matthew T Stahl, and Mark A Murcko. Virtual screeninga an overview. Drug discovery today, 364):160-178, 1998.
Qiang Wang, Yanhao Zhang, Yun Zheng, Pan Pan, and Xian-Sheng Hua. Disentangled representation learning for text-video retrieval. arXiv preprint Sttt::2203.0711111 2022.
David Weininger. Smiles, a chemical language and information system. 1. introduction to methodol- ogy and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36, 1988.
Jun Xia, Lecheng Zhang, Xiao Zhu, Yue Liu, Zhangyang Gao, Bozhen Hu, Cheng Tan, Jiangbin Zheng, Siyuan Li, and Stan Z. Li. Understanding the limitations of deep models for molecular property prediction: Insights and solutions. In Thirth-seventh Conference on Neural Information Processing Systems, 2023. URL pestpps://pperevvviie net. forum d=NLF q Deuzt
Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. Hierarchically structured meta-learning. In Proceedings of the 36th International Conference on Machine Learning, 2019.
Geyan Ye, Xibao Cai, Houtim Lai, Xing Wang, Junhong Huang, Longyue Wang, Wei Liu, and Xiangxiang Zeng. Drugassist: large language model for molecule optimization. arXiv preprint arXiv:2401.10334 2023.
Qiying Yu, Yudi Zhang, Yuyan Ni, Shikun Feng, Yanyan Lan, Hao Zhou, and Jingjing Liu. Mul- timodal molecular pretraining via modality blending. In The Twelfth International Confer- Learning Representations, 2024. htt/ttts:///perre//// weteeeiwwwwwwweeeeeeeeee...eeeee wwtp:neeetfooeooooooooo forum?id-
Jbxck
Chaohao Yuan, Songyou Li, Geyan Ye, Yikun Zhang, Long-Kai Huang, Wenbing Huang, Wei Liu, Jianhua Yao, and Yu Rong. Functional protein design with local domain alignment. arXiv e-prints, pp. arXiv-2404, 2024.
Chaohao Yuan, Kangfei Zhao, Erean Engin Kuruoglu, Liang Wang, Tingyang Xu, Wenbing Huang, Deli Zhao, Hong Cheng, and Yu Rong. A survey of graph transformers: Architectures, theories and applications. arXiv preprint 2xXx::2502./65:. 2025.
Wang Zeng, Sheng Jin, Wentao Liu, Chen Qian, Ping Luo. Wanli Ouyang, and Xiaogang Wang. Not all tokens are equal: Human-centric visual analysis via token clustering transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11101-11111, 2022a.
Zheni Zeng, Yuan Yao, Zhiyuan Liu, and Maosong Sun. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals. Nature communications, 13(1):862, 2022b.
15

---

Published as a conference paper at ICLR 2025
Appendix
A COMPARISON TO RELATED WORKS
The primary challenge in multi-modal representation learning is effectively leveraging information from various modalities to learn common representations shared between them, i.e., aligning dif- ferent modalities. Building on the success of multi-modal representations in the vision community, exemplified by models such as CLIP(Radfom) vancements have found widespread applications in the life sciences, including domains such as small molecules (Luu et al. 2023c), proteins{ Yuan al. 2024). and materials(Ock et al. 2024).
al. 2021b) and BLIP et al., 2022). these ad-
Existing multi-modal molecule alignment approaches can be broadly categorized into two types: internal modalities and external modalities. Internal molecule structure representations include ID fingerprints and molecule strings (specifically, SMILES Simplified Molecular Input Line Entry (ystem)(Weiningg 1988), 2D topological graphs Yuan al. (025). and 3D conformational structures(Li al.) 2024b). External functional descriptions encompass textual descriptions and biological knowledge graphs.
A.1 TEXT GUIDED CONDITIONAL MOLECULE GENERATION
Text-based molecule generation models can be primarily categorized into two types. One type uses a decoder-only transformer architecture, such as MolXPT (Luu et al. 2023d). This is GPT-like model that utilizes the GPT-2medium configuration, which has been pre-trained on SMILES sequences encapsulated by text. The second type employs type translates between text and molecule strings, and it can adapt to the text-conditional generation. Models like MolT5 and Text+Chem T5 (Christofidellis et al. 2023) work by jointly encoding the molecule string and natural language. They then use the input description to generate a molecule string.
rrooder-decoder transformer architecture. This
novo
In Table 10 we provide a comprehensive overview of existing works on molecule-text alignment methods. We have identified two key distinctions from existing methods: (i) Current contrastive bearning-based alignment methods primarily align global features, neglecting finer-graine modal interactions. Fine- aineggrined alignment important tasks such controlled molecule generation and molecular captioning, as it enables greater precision and accuracy. (ii) Existing end-to-end training methods use conditional generation to create molecules, without aligning the molecule modality and text modality. However, our experimental results demonstrate that performing alignment before conditional generation can significantly improve generation performance.
Table 10: Comparison between Atomas and existing molecule-and-tex cross-modal methods.
Training
Model
Text
TD
Malcole
20
Alignment
Midrentte Ratrinall
Malernta Generation
Matrentle Caption
Mutti-Stapr
End-tt-Fad
McMu
Ayppy
KV-PUM
InstrecMoo
10 R014
MrNKA
Ma11
ThateChem T Crrenii certeeeeess 2023
McXXP
MalReeiP
MovenkeTT
AsmaxtOuir
A.2 DIFFRENCE WITH MOLECULAR MODALITIES REPRESENTATION LEARNING METHODS
Different alignment objectives: mole-laa modality Extra-molecular modality. Feng et al. 2023 Yu et al. 2024 Di et al. 2022; Liu et al.) 2022) focus on aligning intra-molecular modalities.
1D SMILES, 2D molecule graph, and 3DD structure, which is different from the
16

---

Published as a conference paper at ICLR 2025
text-based molecule representation learning. This alignment benefits from easily obtainable molecular datasets and pair-wise alignments, facilitated by tools like RDKit for SMILES-to tottoo-pphhhoo conversion. Conversely, Atomas addresses text-driven tasks, involving cross cross-modal learning between intra- and extra-molecular modalíties. Challenges arise from the lack of expert fine-grained textual annotations for molecules and difficulty in constructing positive/negative pairs, text fragment may suit multiple molecule substructures. These challenges make Atomas' achievements in this field particularly noteworthy.
Different segmentation objectives: (Feng et al. 2023) utilizes established algorithms for the segmentation of SMILES and graph representations. In contrast, Atomas the first segment both text descriptions and SMILES.
Different fine-grained alignment methods: Explicit vs Automatic. Atomas uses Hicrarchical Adaptive Aligmment, eliminating the need for explicit labeling between text fragments and molecular substructures. The weighted alignment allows for flexible representation of cross-modal rela- tionships. This approach is particularly beneficial in scenarios where the alignment between text and SMILES is not direct or where the textual data is rich in contextual information that requires sophisticated modeling.
Different training objectives: Prediction task vs Understanding and Generation task (Feng et al. 2023; Yu et al. 2024: Ji et al. 2022, Liu et al., 2022) are tailored for prediction tasks, while Atomas optimizes aligned representations for both understanding and generative tasks.
Table 11: Comparison between Atomas and existing molecular modalities representation learning methods.
Model
Text
HSSMILLS
Milboale Zh-Graph
3D JD-Stnurtare
Antormadi
Expikit
Protiction
Matocule Retrival
Melecale Generation
Matecale Captin
MOI FRE RU 1
ReLMSae
Aboois (00PS)
B PRELIMINARIES
B. MOLECULE REPRESENTATION
The structure of a molecule can be represented as a ID molecular string. Specifically, SMILES is utilized to convert a chemical's 's 3D structure into a string of symbols. For instance, the structure of benzene ring can be represented as a SMILES string: C/=CC=C((=CI) The 1D molecular string and the 2D) graph are informationally equivalent, as SMILES can be losslessly converted to graph using chemical toolkits (c.g. RDKit). Furthermore, transformer-based encoder models exhibit less information loss compared to Graph Neural Networks (GNNs) (Ma et al. (022). which suffer from over-smoothing problems, and GNNs cannot perform unified encoding with text modality, posing challenges for interaction between the two modalities. In this study, choose the 1D SMILES string and textual description for molecule cross modall representation learning.
B.2 ENCODER-DECODER T5 LANGUAGE MODEI.
T5 (Raffel al. 2020) is an encoder-decoder transformer-baser model. In the self-supervised pre-training stage, for the input sequence X, some words in the sequence are randomly chosen for corruption. Each consecutive span of corrupted tokens is masked by sentinel token(e.g- < x >,< y >). Then the objective is to reconstruct the dropped-out spans:
Lyntm (X:0)
Σ
log (xg Zje mask 6)
2o
mask
We use 12-layer T5 model the backbone and initialized using MolT5 (Edwards et al. 2022 weights, which have been pre-trained on both the textual modality C4 dataset (Rafeel et al. (0200) and the molecular modality ZINC dataset (Sterling & Irwin 2015) in self-supervised manner.
17

---

Published as a conference paper at ICLR 2025
B.3 DENSITY PEAKS CLUSTERING ALGORITHM
The Density Peaks Clustering Algorithm (DPC) (Rodriguez & Laio 2014) is granular computing model that determines the number of clusters and their respective centers in a dataset by identifying density peaks. The algorithm is based on two assumptions:
Assumption I.
The local density of cluster centers (density peaks) is greater than the local density of their surrounding neighbors.
Assumption 2. The distance between different cluster centers is relatively large. Briefly, given dataset D {01 -2.-.... J with n samples, the local density of z is defined as - ΣØ= x(day - de). where x is an indicator function: xxx 1 when æ < 0, and x(x) O otherwise, and de a cutoff distance. The relative distance defined as 6 minj.p;>pp (d.z). Based on Ps and DPC algorithm constructs decision graphs to classify data points z; as density peak points, normal points, or outliers.
18

---

Published as a conference paper at ICLR 2025
C LIMITAIONS
As Atomas demonstrated excellent generation and retrieval capabilities, there may concerns about potential overfitting. To validate the generalization ability of Atomas, used the nut-of-Distribution (OOD) dataset, PCdes, in the retrieval task. Atomas outperformed recent state-of-the-ar methods in both lexl-to-molecule and molccule-to-text retrieval tasks &@ indicating robust generalization.
D ALGORITHM
Algorithm The proposed hierarchical cross-modal molecular representation learning framework that jointly learns representations from SMILES and text.
1: Input: SMILES strings S, Text descriptions T
2: Output: Aligned representations molecules and texts
3: procedure UNIFIEDENCODER(SS T)
4: Sembed f Encode S)
5: Tembed Encode
6: return Sembed; T Tembeed
7: end procedure
8: procedure ADAPIIGPOLYMMERZATIOGG
Tembed)
9:
for each level in (atom, fragment, molecule) do
10:
Sclustered &- (luuterrTokessSSmllll (Sembed))
Telustered t- ClusterTokens/ (Tembed)
12:
Sembed; Tembed +- MergeClusters, Selustered: T Telustered)
end for
14: return Sembed, Tembed
15: end procedure
16: procedure
T Tembeed
17:
for each level (atom, fragment, molecule) do
18:
alignment +- ComputeAlignmen (eeeeenttttmmeeee (Sembed, Tembedd)
Sembed, Tembed <- UpdateRepresennations/ (Sembed, T Tembed; olignment) 20: end for
19:
21: return Sembed, Tembed
end procedure
23: procedure CONDITIONALDECODERR (Smbeed T Tembedd
GeneratedOuttput Decode Sembed, T Tembedd
25: return GeneratedOutput
26: end procedure
27: Begin
28: Sembed, Tembed + UNIFIEDENCCODER(S.. (SeN Sembed, T Tembed)
29: Sembeds Tembed ADAPTIVEPLYYEENZAATOO
30: Sembed, T Tembed + WEIGHTTDAALIGGMEETTMOODUU S (Sembeed Tembed)
31: Output CONDITIONAL CeADCCODDERRSeeeeee (Sembed, T Tembed)
32: End
E DATA DETAILS
E. INITIAL TRAINING DATASET CONSTRUCTION
We obtain dataset of 280K molecule-text pairs from PubChem database and follow the Molec- ularSTM to preprocess the textual descriptions, named PubchemSTM-ran Molecule names are replaced by "This molecule is ... ..."or "These molecules are ..."to prevent the model from identifying molecules by name alone. To create unique SMILES-text pairs, molecules with the same CID (Chemical Identifier) are combined, resulting in 243K pairs. Text descriptions with less than 18 characters filtered out, resulting in set of 64,285 samples. In order to avoid data leakage, removed duplicates from the ChEBI-20 and PCdes datasets used in downstream tasks. Specifically, first convert the smiles string into canonical SMILES string using the RDKit toolkit, and then
19

---

Published as a conference paper at ICLR 2025
de-duplicate the initial training dataset with the same SMILES string the ChEB1-20 dataset and PCdes datasets, respectively. This resulted high-quality and leak-free dataset 51,340 pairs, named PubchemSTM-dae It should be noted that PubchemSTM-disee is used exclusively for initial training and does not divide the training, testing, and validation sets.
E.2 DATASET STATISTIC
Table presents the dataset statistics. We observe that PubchemSTM-raw includes many uninforma- tive texts; for example, some descriptions include just one word, such as "4,4"-Methylenebis" and one molecule corresponds to multiple descriptions. So, we first create unique SMILES-teex pairs by grouping pairs by CID, and then filter out the texts with length of less than 18 characters.
Table 12: Statistics of the datasets.
Dataset
Molecule- Text Pair
Train
Valid
Min Word
Avg Word
Median Word
PubchemSTM-reze
280011
18.37
13
PubchemSTM-- TM-disdl
51340
51340
18
44.64
30
ChEBI-20
33008
26407
3301
3300
18
43.49
40
PCdes
14995
10495
1500
3000
17
61.2
50
E.3 DATASET EXAMPLES
Figure shows some examples of our dataset. To make it casier to understand, use the RDKit toolkit to convert SMILES strings into a 2D molecular graph. As shown in Figure [8] the text descrip- tions available to contain detailed local descriptions of molecular structures. The corresponding parts of the text and molecular structures are highlighted with the same color in the table for clarity. Effectively utilizing these localized descriptions is crucial for enhancing the performance of text- based controlled molecule generation tasks. The result of our experiments offers substantial evidence supporting the significance of leveraging these fine-grained descriptions in the generation process.
Molecule Graph
SMILES
Description
The molecule biflavonoid that is obtained by oxidative coupling of two moleccles of apigenin resulting positions C-6 and C-8 the two chromene rings. has role as an antineoplustic agent, an antiviral agent, a hepatoprotective agent and a metabolite. dydroxyflavoor and biaryl.
C-005-0066-0--00--0000000
bond between
Ojo
biflavonoid, a
CI-CCI-CCC-CCCCIC--00N00
The molecule hydrocyphenylalating canying hydroxy substituents positions and of the benzeme ring. It has role as
H0
human metabolite, It is tyrosine derivative non- protppoocinoeecicc alpha-amino acid.
HO
hyyroxyphenylalanne
NH2
CCL-ONICCCCCCIMMCOONNCC C-C1C-2200C1111
The molecule is a phenylurea that is urea substituted by 1-acety hicett/piperidin-444 and 4- (uifluoromethoxy) Jphenyl groups at positions respectively, It has as an EC 3.3.2, (soluhle epoxide hydrolase) inhibitor.
Figure Examples of dataset.
20

---

Published as a conference paper at ICLR 2025
F INITIAL TRAINING DETAILS
F.1 TRAINING SET UP
For the backbone models, we choose MolT5-base which is pre-trained in masked language modeling manner on two uni-modal datasets: natural language dataset and molecular dataset. For the model implementation, we use the PyTorch Lightning framework and employ distributed parallel training. We use the AdamW optimizer with weight decay and the learning rate is set to le The key hyperparameters used Atomas are illustrated in the table Table
Table 13: Hyperparameter details for Atomas.
Hyperparameter
Value
batchsize
16
epoch
100
encoder learning rate decoder learning rate
le-4
le-4
text projection learning rate molecule projection learning rate max padding length queue size precision
le-5
le-5
512
13200
BFloat16 Automatic Mixed Precision
F.2 TRAINING ANALYSIS
visualized the loss curve for training 100 epochs during initial training. We scaled the loss value using logarithmic scale. Figure 9alshows
FigureDalshows the loss curve of the global alignment g after the addition of hierarchical adaptive alignment loss Choa- Figure 9b shows the loss curve of language modeling fim after the addition of the hierarchical adaptive aligmment loss Lrea. The observations suggest that the hierarchical adaptive alignment enhances global alignment and controllable generation.
+ has
w/o
w/ol
110
10003
1509] 20009
25003
30003
10080
20000
(a) The convergence of Lya loss both in the absence and presence of Chau loss.
(b) The convergence of Cim loss both in the absence and presence of Chon loss
21

---

Published as a conference paper at ICLR 2025
G DOWNSTREAM TASK DETAILS
G.1 METRICS
Metrics of Molecule-Text Retrieval: In retrieval task, we are consistent with the natural scene, using the most common search indicators the natural scene (i.e., text-image retrieval (Radford et al. (021a)). Recall at 1/5/10 (Manning et al.] 2008) is a performance metric for information retrieval systems, such as search engines or recommendation systems, that measures the proportion of relevant results found within the top I, 5, or 10 returned items, indicating the model's effectiivnesss in retrieving pertinent information. MRR Voorhees 2000): mean reversed rank. MRR evaluates information retrieval model by averaging the inverse positions of the first relevant results across multiple queries, reflecting the model's 's effectiveness in ranking relevant items.
Metrics of Molecule Captioning: In molecule catpioning task, we follow the MolT5(Edwards et al. 2022) model and employ BLEU (Bilingual Evaluation Understudy) (Papineni et al.] 2002) and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) (Lin 2004) as evaluation metrics for captions. BLEU measures the overlap of n-grams between the generated text and the reference text. BLEU-n refers to the BLEU metric with n-grams, where n is an integer value (e.g.. for unigrams, 2 for bigrams, 3 for trigrams). For example, BLEU-1 measures the accuracy of word level, and higher-order BLEU can measure the fluency of sentences. The BLEU score ranges between 0 and 1. A BLEU score of 0.6 or 0.7 is considered to be a good result. ROUGE-N measures the overlap of N-grams (e.g., unigrams, bigrams, trigrams) between the generated and reference texts. calculates precision, recall, and F-score for N-grams, providing a balanced assessment of the model's performance. ROUGE-L is based on the longest common subsequence (LCS) between the generated and reference texts. It considers the longest continuous sequence of words that appear in both texts, capturing the overall coherence and flow of the generated text.
Metrics of Text-based de Novo Molecule Generation: In molecule generation task we BLEU, Exact, Levenshtein (Miller et al.) 2009). MACCS FTS (Durant et al., 2002). RDK FTS (Schneider al. 2015), Morgan FTS (Rogers & Hahn 2010), and Validity 7 metric. Exact refers to the Exact Match metric measures the percentage of predictions that exactly match the true labels. The Leven- shtein distance, also known as the edit distance, is a metric used to measure the similarity between two strings by calculating the minimum number of single-characte edits (insertions, deletions, or substitutions) required to transform one string into the other. Validity refers to the percentage of molecules that can be processed by the RDKIT and measures the grammatical normality of the generated molecules. MACCS FTS, RDK FTS and Morgan FTS are molecule fingerprint metrics.
G.2 BASELINES
Baselines of Molecule-Tex. Retrieval: MolFM (Luo et al. 2023) jointly trains three unimodal encoders, which separately encode molecular structures, biomedical texts, and knowledge graphs to learn joint representations. MoMu (Su et al. 2022 is pre- mre-trained model that utilizes contrastive learning to align molecular graphs with their corresponding text descriptions. MoleculeSTM (Liu al. 2023c> designs multi-modal contrastive learning model for molecular understanding that incorporates both molecular structural information and textual knowledge. MolCA (Liu et al., 2023e) enables language models to understand both text- and graph-based molecular contents via the cross-modal projector.
Baselines of Molecule Captioning: We compared 9 baselines including MoMu, MolXPT, GIT-Mol
(Liu et al. 2024), MolFM, MolTS, MoIReGPT, Text+Chem T5, InstructMol (Cao et al. 2023) and MolCA (Liu et al. 2023ch. MolXPT is GPT-like model that uses the GPT-2medsum configuration pre-trained on SMILES sequences wrapped by text. GIT-Mol maps molecular graphs, images, and text SMILES modalities into a unified latent space using a GIT-Former designed based on the Q- Former architecture in BLIP2 (Li et al., (023b)). MolTS is a T5-based text-to-text model, pre-trained on large-scale single-modal corpus of natural language and molecules. thereby obtaining prior knowledge of the two domains. MoTReGPT employs GPT-3.5-turbo and GPT-4-0314, and designs retrieval- based prompt paradigm through in-context learning to improve molecule discovery without any additional training. Text+Chem T5 develops multi-task, multi-domaii model for natural and chemical language. InstructMol employs instruction-tuning manner through two-stage training to fine-tune I.LMs (large language models). InstructMol+GS refers the of both molecular graph
22

---

Published as a conference paper at ICLR 2025
tokens and SMILES tokens input. MolCA bridges molecular 2D graph and text by projecting the graph into a semantic space similar to text.
Baselines of Text-based de Novo Molecule Generation: We compared the performance of six out of seven baselines used molecule generation task. We excluded the InstructMol method from the comparison, is not directly applicable to the molecule generation task.
G.3 SUPPLEMENTARY EXPERIMENT FOR TEXT-BASED DE Novo MOLECULE GENERATION TASK ON PUBCHEM324K DATASET
We conducted an additional experiment to compare Atomas' performance with the satte-of-the--o model, ICMA (Li al. 2024a). on the PubChem dataset provided by MolCA (Liu et al. 2023e). Specifically, we trained Atomas-large using only the PubChem324K training dataset, aligning the experimental settings with ICMA by ignoring pretaining data. The results are presented in the Based the data presented, Atomas achieves significant improvements on most metrics over ICMA, including a 63.1 11% improvement on the BLEU metric and a 46.54% improvement on Levenshtein compared to ICMA. This demonstrates Atomas' superior performance on the PubChem dataset, highlighting its robust generalization capabilities.
Table 14: Performance comparison of text-based de novo molecule generation on PubChem324k test dataset. Bold and underlined indicate the best and second-ber results, respectively. "f" denotes that higher is better. " denotes that lower is better.
Model
Model sizes
BLEU
Levenshtein]
MACCS FTS!
RDK FTS)
Morgan FTS
Validity
ICMA(Galactica- MicGGalcttca-12MMaaaaa 2044
125M
0.569
52,75
0.719
0.579
0.652
0.825
ICMA(Mistral.77B)) 2244
7B
0.450
77.01
0.764
0.624
0.504
0.891
Atomas-large
825M
0.734
28.186
0.773
0.637
0.535
0.945
H ABLATION STUDY
H.1 THE SCALABILITY AND ROBUSTNESS OF ATOMAS
Tables 15 and 16 show Atomas perform on complex molecular structures and textual descriptions. "Mol_len 100" and "Text_len 100" indicate lengths from 100 to 200. Atomas performs more robust performance than baseline methods on complex molecular structures and highly technical textual descriptions.
Table 15: Performance comparison on complex structures using CHEBI20 test dataset on molecule caption task.
200
1450 255
(U23)
01722
[641
1.4
41224
01200
0770
E33
(118)
11211
E35
13355
700
1116
0350
0620
8412
0.32
053)
0607
100
0050
180
0134
140
0.513
0142
Table 16: Performance comparison on complex textual descriptions using CHEBI20 test dataset on generation task.
Tea Allarn B1A01
Seaithiem aapa BEAST
Mac 61AST
Avmme Shoopea FEXT
Tial- www.rr/raaaaaaaa FTNT
Malt Margue FIN
1156
100
0.711
3-211
05
0111
535
08
088
178
0228
31.00
21 200
181
6.0
1.00
Tables 17 and 18 presents the complete results the two scaling experiments to show Atomas's scalability and robustness.
23

---

Published as a conference paper at ICLR 2025
Table 17: The scaling of the initial training dataset on molecule generation task Here we choose 15k training data to consistent with MolFM which is one the baselines in the paper.
Model
Data sizes
BLEUT
Exact
Levenshtein
MACCS FTST
RDK FTST
Morgan FTST
Validity
Atomas base
0
0.854
0.298
13.47
0.898
0.809
0,750
0.947
Atomas-buse
15k
0.861
0.318
14.68
0,902
0.817
0.757
Atomas-base
51k
0.868
0.343
13.76
0,908
0.827
0.773
0,971
Table 18: The scaling of the model size on molecule generation task The results presented below demonstrate that increasing the parameters of Atomas can lead to further improvements generation performance.
Model
Model sizes
BLEUT
Exactt
Levenshtein
MACCS FIS!
RDK FIS:
Morgan FTS?
Validity
MolkeGPT (GPT S...:.3.--- S-ttu-u
>173B
0.750
0.1.9
24.91
0.847
0.708
0.624
0.837
MolReGiP (GPT-4-04133
>1758
0.857
0.280
17.14
0.903
0.805
0.739
0.899
MalT3-larg
783M
0.834
0.318
16.32
0.889
0.813
0.750
0.958
Atomas-bass
271M
0.868
0.343
13.76
0.908
0.827
0,773
0.971
Atomas-large
H.2 ABLATION STUDY FOR UNIFIED ENCODER VS SEPARATE ENCODER
Table 19 presents the advantages of utilizing a unified encoder in the Atomas.
Figures 10a 10d illustrates the distribution of the ChEBI-20 training data after weighted sampling. These data is utilized to verify that the unified encoder outperforms separate encoders in scenarios with limited data availability.
Table 19: Ablation study for the use the unified encoder and separate encoder. Performance on the text- de novo molecule generation task using the ChEBI-20 dataset. "f" denotes that higher better. "4" denotes that lower is better.
Method BLEU
Exact
Levenshtein
MACCS FTS
RDK FTSt
Morgan FTSt
Validity
Baseline 0.783
0.082
24,846
0.788
0.661
0.602
0.787
Sep-encoder
0.853
0.278
15.72
0.895
0.805
0.745
0.945
Uni-encoder
0.854
0.298
15,472
0.898
0.809
0.750
0.947
24

---

Published as a conference paper at ICLR 2025
1750
2000
1240
Frequoccy
1500
1000
1000
750
500
25
100
125
150
25
75
100
125
159
Text Length
Text Length
(a) The text length distribution of full ChEBI-
(b) The distribution of text length in 75% of the ChEBI-20 training dataset after applying weighted sampling.
20 training dataset.
500
1000
400
Froqoncy
Pequeay
300
400
200
100
0-
25
100
125
150
25
75
100
125
150
Text
(c) The distribution of text length in 50% of the ChEBI-20 training dataset after applying weighted sampling.
(d) The distribution of text length in 25% of the ChEBI-20 training dataset after applying weighted sampling.
25

---

Published as a conference paper at ICLR 2025
H.3 ABLATION STUDY FOR THE EFFECTIVENESS OF COMPONENTS.
Table 20 presents the complete results of the ablation study for the effectiveness of components.
Table 20: Ablation study for the effectiveness of components. Performance on the text-based de novo molecule generation task using the ChEBI-20 dataset. "f" denotes that higher is better. "!" denotes that lower is better.
Ago
Cnco
Cim
BLEU
Exact
Levenshtein,
MACCS FTS?
RDK FTS!
Morgan FTS!
Validity
0,783
0,082
24.846
0,788
0.661
0.602
0.787
0.841
0.223
16,946
0.886
0.784
0.716
0.954
0.844
0.266
16.675
0.893
0.799
0.736
0.952
0.854
0.298
15.472
0.898
0.809
0.750
0.947
H.4 ABLATION STUDY FOR THE EFFECTIVENESS OF JOINT OPTIMIZATION.
Table presents the complete results of the ablation study for the effectiveness joint optimization.
Ablation study for the effectiveness of joint optimization. Performance on the text-based de novo molecule generation task using the ChEBI-20 dataset.
Table 21:
Method
BLEUT
Exact
Levenshtein
MACCS FTS!
RDK FTS?
Morgan FTS
Validity
Baseline 0.783
0.082
24.846
0.788
0.661
0.602
0.787
2Stages 0.782
0.106
26.029
0.812
0.689
0.602
0.910
Jointly optimization
0.841
0.223
16.946
0,886
0.784
0.716
0.954
H.5 ABLATION STUDY FOR THE EFFECTIVENESS OF DIFFERENT NUMBER OF HIERARCHICAL ALIGNMENT LEVELS,
Table 22 presents the complete results of the effect of different number of hierarchical alignment levels.
Table 22: Ablation study for the effect of different number of hierarchical alignment levels.
Performance the text-based de molecule generation task using the ChEB1-20 dataset.
Level Num
BLEU
Exact
Levenshtein|
MACCS FTS1
RDK FTSt
Morgan FTS
Validity
0
0.841
0.223
16.946
0.886
0.784
0.716
0.954
2
0.854
0.289
15.506
0.896
0.805
0.746
0.950
0.854
0.298
15.472
0.898
0.809
0.750
0.947
0.852
0.287
15.580
0.897
0.808
0.746
0.952
H.6 JOINT OPTIMIZATION BENEFITS BOTH LEARNED REPRESENTATION QUALITY AND GENERATION TASK PERFORMANCE.
The essence of joint optimization is the mutual facilitation between the caption/generation task and molecular representation learning (retrieval tasks). Here we provide the in-depth discussion of this joint optimization strategy. From model perspective: As suggested by the existing study (Bahdanau et al. 2015), attention- based generation tasks essentially perform form of soft alignment. During the generation process, the attention mechanism facilitates mutual translation between text and SMILESS reinforeing the semantic consistency between the textual description and the molecular structure it represents. Concurrently, representation learning bridges the domain gap between text and SMILES, enhancing the caption/generationn task. From data perspective: The captioning and generation tasks may provide complementary information for learning molecular representations. These tasks necessitate the model learn the mapping between text and molecular domains, which allows the model to grasp the intricate relationship between textual descriptions and molecular structures, thereby enriching the quality of the learned molecular representations- taaresetttions--hhrrrhchccllll hierarchical alignment further aids in capturing local data pair relationships, benefiting the generation process.
26

---

Published as a conference paper at ICLR 2025
1 HUMAN EVALUATION
To provide additional validation the model's performance and practical utility, we incorporated human expert evaluations. We randomly selected five molecular captions of varying lengths generated by Atomas, Tex++Chem T5, and MolT5. The average rankings from these evaluations are shown in Table 9 where a lower average ranking indicates better performance. Atomas achieved the overall best performance among the three models, ranking first in 4 out of 5 generated molecular captions.
Below are the SMIELS samples tested by human expert:
SMILES(D: [CCC@))cCCCCeH)CCCC CAC.A (=0)CCSC))/CCICC (CCC://CCCCCCCCCCC0000 (H(CC)))))
SMILES(2: CIC@ CHCCOONCCHICCCCOOOCCCCCCCCCCCCOO
C(=O)OJNC(=O)CC& ( @HI.DOO@@eHIICCCCCCCCCCCCCCTTTTC
SMILES(3): CCTEREATTCCCEECCCCTTTTTTTCCOOCCC
@H]50)0)0)CCC ([C@H)(C@@ CHCCCCCCCCCCCOCCCCCCCCCCCCCCCCCC ocC 0J0)0JO
SMILES(4);
SMILES(5):
([C@ @ BICCTCCTTTOOOOOOOOOOCCCCCCCCTTTC ([C@H][C@@ COOONNCCDSSCEOOOCTTTTTTTTTTCCEEE ([C@H)([C@@))) (eCelCCC@H(CCCCCC BHCCHHOOOOOOOOOOOOCCCOOOOOOOOOOO
@H]
27

---

Published as a conference paper at ICLR 2025
J
CASE STUDY OF MOLECULE CAPTIONING
Figure 11 shows more molecule caption examples selected from ChEBI-20 test dataset.
Molecale Input
Atomas Output
Groand Truth
The molazule is an unsitumtedd fulty acys-CoA that results from the fommal cumsensation of the thiol group coeuzyme 8.9-ejooy-(57, 117, scosaarnneii seid. is long-chuin filly acyl-CoA and unsatunted faily acyl derives an 89-EET. coujiagntc scid of as 8, 8.9-epoxy-557,1 112, 142)- ioostrionnyl CooA)))
The melecule on unsatmnted Eatty seyl-Coo that results from the furmal condensation of the thiel group caenzyme epoxy-(57, 112, 1-7))kcostttienoic scid. is long- chain filly scyl-CoA and unsatumited fiily anyl- Cor. It derives from an 8.9 acid of an 9-spoxy- (52 112, 44/-iiesttiiisoy CoA(4-).
the carboxy group of
the carboxy group
conjugate
BLEU-2: 1.0
The molecule organophoophste uxcanion obtained by depcotonation of the carbuxy and phosphate OH groups of (98, (0R)- (1-bydroxy--- aekoophonoosyootteeeeee
melecule organophosphatt ovaanion obtained by deprotenation of the carboxy and phosphate OH groups of (98, 10R 10R)-100 10-hyrrorr. (phoogkensoorry jectidecancic acid; major spocies pH 7.5. is on organohhoppatc oxnanion hydrory monocarbusyliic scid anion. it base of (05))0R) 10-bydrosy 9- (phoolicnoxxyeereeeee joooooooooooodooooiic soid.
acid; major species pH 7.3. an orgenephosphat exoosion and monocirbungllic acid anion. It is conjugate base of (95, LOR) 10 (0-yyrroyy 9-
BLEU-2:000 0.95
(phosphonoory cocooooyoootttdoooo joctodecanniic acid.
The molscale a methyl glyeaside that consists of 30-55-aminupenty Dasplaaa----mmmmm and three N fumayl sipha D perosamine residues linked sequentiallyy (1 (1-23) (1-33) and (1-22) and linked the relneing and glycosidically to methyl group. methyl glycoside and a trisacchuzide desivarive.
The molecule is methyl glycoside that consists of 4-O+ -aminupentyD--iii D-aalpa------mmmeee D-momnose residue and two formy) alpha perosamine residues linked sequentially (1=2) and (1-23) and linked at the relucing end glycosidienlly to methyl group. methyl glycoside and a trsaccharide derivative.
residue
BLEU-2: 0.9
The malecule the or organosulionnt axcanion that trionion Resetive Bhe formed by loss a proton from each the sulfo groups: major species pH 7.3. is conjugate base of Reactive Bluc
The melecule the organosuiionate excanion that is the rrianion Renetive Blue quinal form, obtained by loss of proton from each of the sulfo groups; major species pil 7.5. It conjugate Blle quinoi forn.
BLBU--2 0.85
The molecale a pyrrlizinne alkaloid that produced by a bybrid species of Jacobaea. It has role Jacsboca metabolite. is pyrrolizine alkaloid, a tertiary amine oxide, a lertarry alcohol, a macrocyclic lactone, an organic heterotrcyylli compoind and testary anine onide. desives from a Senceivering,
The molecule a perroliziee alkaloid that jacoline in which the tertiney amino funetion has been oxidiaed the correspanding N-oside. It has role as Jacobuea metaboliie. is mucrocyclic lactone, an organic beterotriycclic compound, a pyrolizine alizaloid, triel and tortary aniine oxids. derives from jacoline.
BLEU-2:0 au
The molecule is pymolizine alkaloid that is produced by bybrid species Jacohaes. role as Jacobaca metaboline. alkaloid, a tertary amine oxidk, a tertary alcohol, a macrocyclic lactone, organic heterotticyclic compound a testirry anine oxide. It desives from a Senceivernine
The mulecule is pyrrolizine alkaloid that jacoline which the sertiary amino function has been oxidised the corresponding N-oxide It role as Iocoure metibolite is a macrocyelic lactone, organic beterotricyclic compound. pynrolizine alkaloid, oside. It derives from a jacoline.
pyrrolizine
tertary aiiine
BLEU-2:0. 0.7
The molecole is hydroty fanty scid anion obtained deprotonation of carbocy function lipuxin A4, major species at pit 7.3. has role a human metabolite and Sacchoromycas cerevisise metabolite. is hydrocy fally acid icosanoid anion, a long-chain furty acid anion and a polyunsaturated fatty scid anion. It is conjugate base liponin A4.
The mulecule a hydrosy fisty acid snioon obtained by deprotomation of carboxy function lipexin A4; major species at pit 7.3. is a hydroxy faity neid anien and a liporin anion. conjugote base of lipuxin
an
BL-UU- 0.6
Figure II: Additional molecule captioning examples.
28

---

Published as a conference paper at ICLR 2025
K CASE STUDY OF TEXT-BASED DE Novo MOLECULE GENERATION
Figure 12 test dataset.
shows more text-based de novo molecule generation examples selected from ChEBI-20
Test Input
Atemas Output
Ground Truth
The molecule carbamate ester thymidine which the 5 s'ayyrosy group has been esterified 3--beroylosy---ae --xoppp--- [eanpropy/Jjaraanic It derives from a thymidine.
Morgan FTS: 1.0
The molocule a resin glycoside tetrasacchardde derivative of jalapinoláe acid. It has isolated from lpoesoes bateras. role as metabolite. is cinnamat ester, a macrocyclic lactone, resin glycoside, dodocancate usten. It derives from a trans cinnamic acid and jalapinolic acid.
tetrassccaride
Morgan FTS:
munoterpenoid
with formula C22H24N204, originallly isolated from the seeds of Strychnos mux-vomica, has nole as mutaboliic. delle- cyclic ketone, monoterpenoid indols alkaloid an organic heterohexacyclic compound, tertirry amino compound
of phenols.
Mongan FTS: 0.9
The molecule as secondary cuboxamide reuilring from the formal condensation of the
cadbxxy group Cernalydro-2H-pyyatt eurhosylie acid with the amino group of 5-([(5- turt-bury butyl 1,3-oxazol-2 2-ximmeihyl]a (aaltlylllllllll)) 1- 1,3- thiazal a-minne
kinase has a role as an BC 2.7. 111.... leyelin-depedeett kinase) inhibitoc. It member of 1,5-oxazales, member 1,3iiinveles, onganic fide, secondary carbuxamide and member of usames.
Morgan FTS: 0.85
The molecule organic heterohesaccclli
compound that cladonismide
which
the chlorimes are replaced by hydrogen. has
isolated from the colture broth of cladoriamiie, onganic heserohexacyclic compound,
Strepteengees uncialis
dicurbotimide, lettary aloohol and diol.
Morgan FTS:
The molocule as polyketide that menacolin bearing an additional hydroxy subscituent at position has
an antimicrobial agent,
a fungal motaboline and an EC LLL.3EEC
HD
HO.
1.1. 1.888 attpooymmtiiggtttrrl..... inhibitor. is sccondary aloohol, polyketide, carbebicyclic compound, member 2-
reductase)
member sesalydrompphiiakkue... mosacolin I.
Morga FTS: 0,7
The molecule a lipid that is lipid A-core which the anomeric phosphate replaced by diphoephate. is member lipid As, dodecsmontic and letralocanosle ester. is conjugate lipid A-core diphoephate(11)) (I.).
Mongan
Figure 12: Additional text-based de novo molecule generation examples.
29


Reviews:
[{"review_id": "TRwzMlV2tu", "reviewer": "Reviewer_1nxB", "summary": "This paper notices that existing works about molecule-and-text representation learning are mainly coarse-grained and uses a global alignment method to integrate molecules and texts. To solve this problem, this paper proposes a fine-grained method for cross-modal representation learning. Specifically, this paper proposes a hierarchical adaptive alignment module, which consists of two components, adaptive polymerization module and weighted alignment module. Experiments on different cross-modal tasks verify the effectiveness of the proposed model. Ablation analysis is also conducted to show the effect of each modeling component.", "questions": "N.A.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Overall, the problem of existing works is clearly motivated in the Introduction section with a figure as visual illustration. The overall writing in the paper is also clear enough, and an algorithm is also provided to formally present the learning process.\n\n2. Specifically, the design of hierarchical adaptive alignment module is interesting and novel. This paper clearly shows its effect with empirical evaluation.\n\n3. Experiments are comprehensive enough with different tasks and metrics. Ablation analysis is also conducted. Visualization also helps understand what the model learns.", "weaknesses": "1. Usually when we do experiments, we encourage authors to repeat the same experimental setting multiple times and report both mean and standard deviation, or report significance t-test. However, some tables and figures in the Experiment section don't have standard deviation or significance t-test, such as Tables 1 and 8, Figure 4. Authors are suggested provide standard deviation in the paper.\n\n2. Scalability is an experiment in the paper to show the proposed is possible to scale on large datasets. For scalability, authors are also suggested to provide computational comeplexity to theoretically show that the proposed model is computationally efficient than baseline models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d3ZO1ISXGX", "reviewer": "Reviewer_N9hg", "summary": "This paper proposed a unified framework to realize the alignment between text and SMILES at three levels without fine-grained manually annotated labels, namely the atom level, fragment level, and molecule level. The combined optimization of fine-grained contrastive learning and autoregressive learning promotes the model's performance on various downstream tasks, such as text-based de novo molecule generation, molecule captioning, molecule property prediction and molecule-text retrieval.", "questions": "1. It seems that the purpose of the Molecule Level Alignment in Hierarchical Adaptive Alignment, and the Global Alignment is the same. Can you explain more about why Global Alignment is still needed in the architecture?\n2. In the Assignment Step, have you tried using other clustering algorithms? How will it affect the results?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.The writing is clear, the diagrams are rich, the experimental results are comprehensive, and it is easy to understand.\n2. This paper realizes the fine-grained alignment of SMILES and text from the atom level, fragment level and molecule level, which is very comprehensive. The visualized results demonstrate the effectiveness of fine-grained alignment.\n3. The ablation experiments demonstrate a mutual enhancement effect: contrastive learning alignment improves autoregressive learning, while autoregressive learning enhances contrastive learning alignment, showing the benefit of joint optimization.\n4. Atomas, relying only on SMILES, achieves superior performance than models that rely on both SMILES and graph on some tasks.", "weaknesses": "1.Previous papers have proved the effectiveness of combined optimization of contrastive learning alignment and autoregressive learning. \nRef.”Align before fuse: Vision and language representation learning with momentum distillation”\n2. The practice of using all tokens to calculate similarity to achieve more fine-grained alignment without additional labels is also not new. And the direct application of the weighted alignment modules limits the innovation.\nRef.”FILIP: Fine-grained Interactive Language-Image Pre-Training”、 “Disentangled Representation Learning for Text-Video Retrieval”.\n3. While the method of constructing hierarchical features in this paper is effective, it lacks a more explicit rationale explaining the underlying principles that make this construction feasible.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sVAxqnnWWS", "reviewer": "Reviewer_1BNJ", "summary": "This paper presents a hierarchical molecular representation learning framework, namely Atomas, which jointly learns representations from SMILES strings and texts. In Atomas, a Hierarchical Adaptive Alignment model is designed to learn the fine-grained fragment correspondence between molecule SMILES strings and text descriptions at three semantic levels, namely atom, fragment, and molecule level. Atomas outperform the baseline models on 12 molecule-related tasks.", "questions": "1. Could the authors compare more advanced baselines?\n2. Could the authors explain my concerns in Weakness 3?\n3. Is it possible to scale this methodology further or apply Atomas to decoder-only models? Currently, it might be unfair to compare Atomas with the previous baselines like MolCA because the backbone LLM can also lead to a difference in performance.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of aligning molecules and texts in a fine-grained aspect is novel and provides insight into the methodology for adopting LLMs for molecule discovery.\n2. The module, Hierarchical Adaptive Alignment, is novel and introduces a new modality alignment method.\n3. The performance of Atomas is competitive and proves its generalization in different molecule-related tasks.", "weaknesses": "1. The description of the framework is inappropriate. The authors claim they align SMILES and textual descriptions in three semantic levels, but the atom level actually uses tokens, which can not be identical to `atoms` as symbols and numbers might exist in the SMILES strings.\n2. In Table 3, the performance of MolCA is not correct. The metrics should be BLEU-2 63.9, BLEU-4 55.5, ROUGE-1 69.7, ROUGE-2 55.8, ROUGE-L 63.6, and METEOR 66.9 with Galac1.3B [1]. This raises a challenge to the authors' claim that these molecule-and-text alignment methods struggle to effectively capture fine-grained correspondence related to local parts within different modalities.\n3. I am a little confused about the three-level alignment. It seems that the alignment still happens at Stage 3, which calculates the similarity between the textual description and the molecule representation. Although the token and token cluster information might be extracted to enhance the molecule representations, I do not see how atom or fragment information is aligned with their descriptions in a fine-grained manner.\n4. The selected baseline models are probably weak. More advanced models like BioT5 [2] [3] and ICMA [4] should be included.\n\n\n#### References\n[1] Liu, Z., Li, S., Luo, Y., Fei, H., Cao, Y., Kawaguchi, K., ... & Chua, T. S. (2023). MolCA: Molecular graph-language modeling with cross-modal projector and uni-modal adapter. arXiv preprint arXiv:2310.12798.\n[2] Pei, Q., Zhang, W., Zhu, J., Wu, K., Gao, K., Wu, L., ... & Yan, R. (2023). Biot5: Enriching cross-modal integration in biology with chemical knowledge and natural language associations. arXiv preprint arXiv:2310.07276.\n[3] Pei, Q., Wu, L., Gao, K., Liang, X., Fang, Y., Zhu, J., ... & Yan, R. (2024). Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning. arXiv preprint arXiv:2402.17810.\n[4] Li, J., Liu, W., Ding, Z., Fan, W., Li, Y., & Li, Q. (2024). Large Language Models are In-Context Molecule Learners. arXiv preprint arXiv:2403.04197.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)