Fragment and Geometry Aware Tokenization of Molecules for Structure-Based Drug Design Using Language Models
Authors: Cong Fu
Xiner Li
Blake Olson
Heng Ji
Shuiwang Ji
Venue: ICLR 2025 Poster (2025)
Keywords: Generative models
language models
molecule tokenization
structure-based drug design
fragment
Primary area: generative models

Abstract: Structure-based drug design (SBDD) is crucial for developing specific and effective therapeutics against protein targets but remains challenging due to complex protein-ligand interactions and vast chemical space. Although language models (LMs) have excelled in natural language processing, their application in SBDD is underexplored. To bridge this gap, we introduce a method, known as Frag2Seq, to apply LMs to SBDD by generating molecules in a fragment-based manner in which fragments correspond to functional modules. We transform 3D molecules into fragment-informed sequences using $SE(3)$-equivariant molecule and fragment local frames, extracting $SE(3)$-invariant sequences that preserve geometric information of 3D fragments. Furthermore, we incorporate protein pocket embeddings obtained from a pre-trained inverse folding model into the LMs via cross-attention to capture protein-ligand interaction, enabling effective target-aware molecule generation. Benefiting from employing LMs with fragment-based generation and effective protein context encoding, our model achieves the best performance on binding vina score and chemical properties such as QED and Lipinski, which shows our model’s efficacy in generating drug-like ligands with higher binding affinity against target proteins. Moreover, our method also exhibits higher sampling efficiency compared to atom-based autoregressive and diffusion baselines with at most $\times 300$ speedup. The code will be made publicly available at https://github.com/divelab/AIRS/tree/main/OpenMI/Frag2Seq.

Published as a conference paper at ICLR 2025
FRAGMENT AND GEOMETRY AWARE TOKENIZATION OF MOLECULES FOR BOUCTURE-BASEL DRUG DESIGN USING LANGUAGE MODELS
Cong Fulx, Xiner Li Blake Olson', Heng Ji,, Shuiwang Ji
Texas A&M University. College Station, TX, USA
2 University of Illinois tibana-Champaigee Champaign. IL, USA /coggfu, 1xe, blakeolson, jij atttamu.de [hengji] (alliiieellnnniissse
ABSTRACT
Structure-based drug design (SBDD) is crucial for developing specific and effective therapeutics against protein targets but remains challenging due to complex protein- ligand interactions and vast chemical space. Although language models (LMs) have excelled in natural language processing, their application in SBDD is underexplored. To bridge this gap, we introduce method, known as Frag2Seq, to apply LMs to SBDD by generating molecules in fragment-based manner in which fragments correspond to functional modules. We transform 3D molecules into fragment- informed sequences using SE(3) eqquivarraa molecule and fragment local frames, extracting (3) -invariant sequences that preserve geometric information of 3D fragments. Furthermore, we incorporate protein pocket embeddings obtained from a pre-trained inverse folding model into the LMs via cross-attention to capture protein-ligand interaction, enabling effective target-aware molecule generation. Benefiting from employing LMs with fragment-based generation and effective protein context encoding, our model achieves the best performance on binding vina score and chemical properties such as QED and Lipinski, which shows our model's 's efficacy generating drug-lit like ligands with higher binding affinity against target proteins. Moreover, our method also exhibits higher sampling efficiency compared to atom-based autoregressive and diffusion baselines with at most 300 speedup. The code will be made publicly available at phtttt::: /tt:://gttttt AARS /tree//eeeeeeeeee http:///s///eeeeeaaaaa//////////
INTRODUCTION
Structure-based drug design (SBDD) is critical method in medicinal chemistry that involves the design and optimization of molecules to interact specifically and effectively with biological targets, typically protein pockets (Anderson) 2003). This approach is fundamental in developing new therapeutic drugs as it allows for more precise interaction with biological systems, potentially reducing side effects and increasing efficacy. Traditionally, SBDD relies heavily on high-throughput virtual screening (Lyne) 2002, Shoichet (2004) and experimental validation. These processes not only time-consuming and labor-intensie but also require substantial financial resources.
In recent times, machine learning has emerged as promising approach for advancing scientific endeavors (Zhang et al. [2023b] [iu et al. (021b), such as physical simulation (Sanchez-Gonzale,
et al. 2020 Helwig al. 2023 Fu al.
2024aa Zhang al. 2024b) Gui 2005), quantum mechanics Koohkov et al. 2202 et al. [2022) materials Yan et al. 2024). and biology (Liu
al. 2021a 2022a Wang al. 2022 Fu al. 2024b). SBDD poses great challenges machine learning models as requires the model to capture complicated protein-ligand interaction while improving the drug-likeness of designed molecules. Earlier works have tried use autoregressive
models (Luo et al. 202 Peng et al. (2022) and diffusion models (Guan et al. 2022 2023 Huang
et
2024b Li et al. 2024b Zhou al. 2024) to encode context information and generate molecules.
"Equal contribution

---

Published as a conference paper at ICLR 2025
However, these models only consider atom-wise generation, and diffusion models typically need thousands of steps to generate, which results in an inefficient generation. Another line of work chooses to generate molecules based on molecular fragments (Zhang et al., 2022 Zhang & Liu 2023), but often requires a complicated pipeline involving several neural networks to choose and link fragments.
Recently, language models (LMs) have demonstrated substantial promise in various fields due to their robust data processing and generative capabilities (Vaswani et al. 2017 Devlin et al. 2018 Brown
et al.
2020 Gu et al. 2021; Wang et al. 20244 Khurana et al., 2024). These models, especially large
language models (LLMs) (Zhang et al. [024a; Touvron et al. 2023 Achiam al. 2023 Chowdhery
ct al.) 2023; Huang et al. 2024a). excel in learning complex patterns and producing coherent
outputs, which makes them ideal candidates for advanced tasks natural language processing and beyond. Despite these strengths, the application of LMs to SBDD remains largely unexplored. LMs offer notable advantages, such as handling large datasets with prominent efficiency over diffusion- based methods, learning from massive biological and chemical texts for diverse potential tasks, and generating molecules and materials (Li et al. 2024a; Yan et al. 2025). However, challenges persist, particularly the need to adapt LMs to process geometric graph structures of molecular data, which are fundamentally different from textual data. Moreover, it's promising to consider how to apply generation, which can generate more realistic substructures and
LMs to perform fragment-base reduce generation steps to improve efficiency. Additionally, ensuring that these models can accurately simulate the physical and chemical properties of molecules remains a significant hurdle, and it's also crucial to consider how to encode protein context information in LMs to effectively capture protein-ligande interaction in order to generate molecules that can bind to target proteins. Addressing these challenges could unlock the transformative potential of LMs in SBDD, leading to more efficient and effective drug discovery processes.
In this work, we propose to employ LMs to generate molecules in a fragment-based manner for the SBDD task. To achieve this, we develop a novel approach to convert 3D molecules into fragment- informed sequences by constructing (3) (3)-euurvari molecule and fragment local frames and then extracting
sequences that contain the {E)3)-invarian -invariant coordinates and orientations of 3D fraagments. To consider protein conditions, to extract the embedding of protein pockets and incorporate this information into LMs by cross- attention mechanism. In this way, our method enables the usage of powerful LMs in target-aware molecule generation while keeping geometric information in sequence conversion. Our experimental results evidence these advantages, outperforming strong baselines in both efficacy and efficiency.
use existing pre-trained inverse folding model
2 RELATED WORKS
Structure-Basd Based Drug Design. Structure-based drug design aims to generate molecules that can bind to the given protein pocket. LiGAN (Ragoza et al. 2022) uses an atomic density grid to represent protein-ligand structures. The atom type, positions, and bonds constructed from generated density
grids. Then, the following works (Luo et al., 2021 Peng et al. 2022 Liu et al. 2022a) generate
molecules using autoregressive models. For example, GraphBP (Liu et al. 2022a) uses normalizing
flow to generate each atom's 's relative position in an equivariant way by constructing local coordinate systems. Pocket2Mol (Peng al.) 2022) also considers bond generation to improve the structure validity, Compared with autoregressive models, diffusion-basee SBDD models (Schneuing et al.)
2022 Guan al. 2022 2023) can generate all atoms of molecules in one-shot way. For example, (Schneuin et al. (2022) denoises all atom positions and types from a Gaussian distribution. IPDiff (Huang et 2024b) uses a pretrained binding affinity prediction model guide the diffusion generation. To generate more realistic substructures, FLAG (Zhang et al. 2022) and DrugGPS (Zhang) Liu 2023) adopt fragment- hagdee methods to generate molecules motif-by-motif. For those methods using language models, TamGen (Xia et al. (2024) only generates SMILES, and Lingo3DMol (Feng ct al. 2024) generates fragment--aas based SMILES then predicts molecule coordinates. They both need pre-training on millions of SMILES or 3D molecule structures to achieve good results.
Language Models for Chemistry. Drawing inspiration from the success of LMs in NLP and beyond, chemical language models (CLMs) emerge competent way for representing molecules (Bran & Schwaller 2023: Janakarajan et al. 2023; Bajorath| (2024] Zhang et al. 2024a). Variants of LMs have been adapted for molecular science, producing a variety of works including DrugGPT (Li et al.
2

---

Published as a conference paper at ICLR 2025
Original molecule
Idantity rotatable bonds
Extract fragment
Figure I: Illustration of splitting molecules into fragments.
2023c),
DrugChat (Liang et al.
2023), MoleculeSTM (Liu et al.
2023a), ChemGPT Frey et al.
2023) MolGPT (Bagal et al. 2021). MolReGPT (Li et al. (223aa) Mol15 (Edwards al. (022))
MoleculeGPT (Zhang et al. 2023a). InstructMol (Cao et al. (023), and many others (Luo et al.
2022 Mao et al. [023b] Haroon al. 2023 Blanchard al. 2023). CLMs learn the chemical vocabulary and syntax used to represent molecules. All inputs including chemical structures and property syntax should be converted into sequence form and tokenized for compatibility with language models. Commonly, SMILES (Weininger) 1988) is used for this sequential representation, although other formats like SELFIES (Krenn al. 2019), atom type strings, and custom strings with positional or property values are also viable options. To learn representations, CLMs are usually pre-trained on extensive molecular sequences through self-supprvised learning. Subsequently, models are fine-tuned more focused datasets with desired properties, such as activity against target protein. Most existing CLM works consider chemical structures as well as other modalities such as natural language captions (Bagal et al. [2221] Li et al.
2202a c |Ewardd et al. [022 Xie et al.
2023]Cheen
2023] Chen et al.
2023b; Tysinger et al. [223] Xu et al. 2023 Chen et al. 2023a Pei et al. 2023
Liu et al. [2023b Wang al. (023). while some Tocus
pure text of chemical literature (Luo
et al. 2022 or molecule strings (Haroon et al. 2023
Mao et al. 2023b Blanchard et al.
2023
Mazuz al. 2023 Fang al. [002]] Kyro al. 2023 Izdebski et al. 2023 Yoshikai al.
2023
Wu et al. [2023] Mao et al. (023a)). Notably, all these works solely consider 2D molecules for
representation learning and downstream tasks, overlooking 3D geometric structures which is crucial in many chemical predictive and generative tasks. In order to use pivotal 3D information, another line of work incorporate geometric models such GNNs in parallel with the CLM (Xia al. 2023
Zhang et al.) 2023a Cao et al. 2023 Liang et al.) [023] Liu et al.) 2023a Li et al. 2023b Frey al. 2023), which requires additional design and training techniques to mitigate alignment issues. However, no existing work uses LMs to directly generate 3D ligands in structure- based drug design.
3 METHODS
In this section, describe how to convert 3D molecules into sequences in fragment-based manner, which can be effectively processed by language models. In Section 3.1 we formally introduce the problem definition of SBDD. Then, in Section 3.2 show how to split 3D molecules into 3D fragments. With these 3D fragments, next in Section 3.3] we introduce the way to construct a bijective mapping between 3D fragments and SF -ii3aiintar sequences by constructing -equiieaaaia molecule and fragment local frames. Finally, in Section 334,, we describe how to incorporate conditional information of proteins into the language model and the training and generation strategies.
3.1 PROBLEM DEFINITION
Our objective is design 3D molecules (i.e., ligands) that effectively bind given protein target within its binding pocket while also demonstrating suitable drug-like properties. The 3D geometry of a protein binding pocket is represented P {(s,, b,) where s; RR3 denotes the 3D Cartesian coordinates of the i-th atom and b, is a one-hot vector that denotes the atom type. Similarly, we represent a ligand as M {(D,, ) where u, R and z, denote coordinates and atom type, respectively. Our goal is to learn a conditional generative model that captures the conditional probability p(M(P) from the training protein-ligand pairs.
3.2 3D MOLECULE FRAGMENTATION
Since we generate molecules in a 3D fragment-based manner, we first need to decompose molecules in the training set into 3D fragments. Specifically, as shown in Figure [ we split fragments by cutting rotatable chemical bonds that meet three conditions: (1) the bond is not in ring, (2) the bond type is single, and (3) the degree of the beginning and end atom on the bond is larger than 1. The third condition prevents breaking the functional groups such the hydroxy and carboxyl
3

---

Published as a conference paper at ICLR 2025
groups. To maintain canonical order of splited fragments, further sort fragments based on the order of their appearance in the canonical SMILES representation. More details of canonical order described in Section 3.3.1 Formally, for molecule M,, we split it into set of 3D fragments {G, = (Z,, vo) where Zi denotes atom type matrix and V; denotes Cartesian coordinates.
3.3
FRAGMENT-BASEI 3D MOLECULE TOKENIZATION
3.3.1
ATOM ORDERING BASED ON 3D GRAPH ISOMORPHISM
As the first step in molecule tokenization, we need to transform a molecular (fragment) graph into ID sequential representation. Thus need establish order for the atoms of given 3D graphs. To acheive dimension reduction with least information loss, we seek uniqueness in the ordering and resort to canonical SMILES (O O'Boyle, 2012: Weininger et al. 1989)as solution. A key property of canonical SMIL JES is its ability to provide a unique string representation for a given molecular structure, which not inherently guaranteed by the basic SMILES algorithm. We refer to set of (atom) orders with canonical properties as canonical orders (McKay et al., 1981 McKay & Piperno 2014). The canonicalization of SMILES involves deterministic process where the algorithm selects unique starting atom and follows set of predefined rules to traverse the molecule in systematic way. This results in a consistent and reproducible ordering of atoms and bonds within the SMILES string, irrespective of the initial input format. The canonical form is crucial for database searches and for ensuring consistency in chemical databases, as it prevents duplicates and allows for efficient indexing and retrieval of molecular data, which ensures maximum invariance for atom reordering.
To formally study which molecules the canonical forms can distinguish between and not, we first give the below definition following graph theories.
Definition 3.1. [3D Molecular Graph Isomorphism] Let M1 = (V1, Z1) and M2 (V2, Z2) be two 3D molecular graphs, where z; is the node type vector and w; the node coordinates of the molecule M;. Let ver(.) denote the set of vertices, attr(-) denote node attributes, and no edge exists. Let M1 = M2 denote two attributed graphs are isomorphic. Two 3D molecules M1 and M2 are 3D isomorphic, denoted as Mi aan M2, if there exists bijection b: ver her(Mi) - ver(M2) such that for every atom in Mi indexed i, z Mi =b(i) and there exists a 3D transformation T E SE(3) such that v; MM =((wM)) ). Ifa small error is allowed such that (o -(w M2 ) S call the two 3D graphs c-constrained 3D isomorphic.
This leads us to the following guarantee.
Lemma 3.2. [Canonical Ordering for 3D Molecular Graph Isomorphism] Let M1 = (V1, V Z1) and M2 (V2, Z22 be two 3D molecular graphs following Def. 3.1 Let L M - L be a function that maps a molecule M E M, the set of all finite 3D molecular graphs, to its canonical order L(M) L. the of all possible canonical orders, as produced by the canonical SMILES. Then the following equivalence holds:
L(M1)= L(M2) Mi 33D M2
where Mi SDD M2 denotes that M1 and M2 are 3D isomorphic.
Lemma 3.2 indicates that the established ordering with canonical SMILES is both complete (sufficient to distinguish non non-3D-isomorpeh molecules) and sound (not distinguishing actually 3D- iiooomorpphc molecules). Detailed proof is provided in Appendix C.I
3.3.2 EQUIAAAANN MOLECULE AND FRAGMENT FRAMES CONSTRUCTION
To integrate 3D structure information into our sequences, one main challenge is to ensure the S F3))- invariance property. Specifically, given a 3D molecule, if it is rotated translated in the 3D space, its 3D representation should be unchanged. In order to incorporate invariance, we first need to construct SE(()-eqqivariant frames for fragments and whole molecules.
Given a 3D molecule M with atom types Z and atom coordinates V. we first build a molecule local coordinate frame m (x. y, 2) based the input molecule. For the fragments (G. of M, we sort the k fragments based on the canonical order L(M), specifically by the order of a fragment's first-ranked atom. We calculate the average atom coordinates as the center of each fragment. As
4

---

Published as a conference paper at ICLR 2025
Pre-trained ESM-IF1
Fragments
Chain of Frames
(Miee my m))))
Protain
Motecule Frame
16.
GPT
Fragment Frame
Cross attention
①
Canonical Order
World Frama
Masked Self- attention
G-CNIINNE-ONC ICCCC00 ICC00C1
Sequance
Molecule
Frugment type,
mar Plg:: Mag""
Tokenization
Spherical coordinate Rotation vector
Figure 2: Overview of Frag2Seq pipeline. Atoms in the small molecule are reordered according to the order in canonical SMILES. Then, we split the 3D molecules into 3D fragments and sort them based on the canonical order. Next, construct S (BE(3))equvvrri which are local coordinate systems for molecules and fragments, respectively. Then, we obtain the -iiiannrtr spherical coordinates (d,0, Φ) of each fragment center under the molecule frame and (3)))-invrrian -invariant rotation vector (mg. My, m. between fragment and molecule frames. Lastly, concatenate them into sequences. Protein node embeddings are obtained using the ESM-IF1 model and incorporated into the language model via cross-attention.
molecule and fragment frames,
shown in Figure 2 the frame is built based on the first three non-collinear fragment centers in the canonical order. Let & l2, and fm be the indices of these three fragment centers. Then the molecule local frame m = (xe, y, z) is calculated as
x normalize(V; ve,), y = normalize ((ve,. Vt x)
y:
where normalize(-) is the function normalize vector unit length. Note that the molecule local frame is equivariant to the rotation and translation of the input molecule.
Given the basis vectors (æ. y, of the molecule local frame, between the molecule local frame m and the world frame w, which we denote as m -> W. Specifically, the transformation involves rotation matrix Rm--w 3x3 with [mm 1 and a translation vector m E IR'. Since the basis vectors (x, y, z) are already normalized and orthogonal to each other, we can directly stack them together to form the rotation matrix, such that Rm- [x=", y², 2²" ". For the translation vector, we can set tm-w Ug, as it represents the displacement between the origins of the two frames.
construct the transformation
Next, we need to build the local coordinate system for each 3D fragment. Similar to the building process of the molecule local frame, we use the first three non-collinear atoms in a fragment to construct the fragment local coordinate frame and we denote it as g. The fragment local frame is also equivariant to the rotation and translation of the input molecule. Similarly, we can obtain the rotation matrix Rg-rw and translation vector fg-w. which represent the orientation and displacement between the fragment local frame and the world frame, respectively.
Under the fragment local frame. we obtain the local coordinates of atoms Vg, E R in the i-th fragment, where g denotes the number of atoms in a fragment. Then, we save each splited fragment from the training and the value to be the atom types and atom local coordinates in each fragment under the associated fragment local frame.
into dictionary with the key to be the canonical SMILES of each fragment
With the transformation from the world frame to the molecule local frame and from the world frame the fragment local frame, and the fragment local frame by using homogeneous transformation conversion. Specifically, we construct homogeneous transformation matrices from rotation matrices R and translation vectors t. Formally, we have
derive the transformation between the molecule local frame
Rm ww 0
Lm 7m-
R
lg w
Tm-rw
1 T
(2)
0
1
where T TRAX1 denotes the homogeneous transformation matrix and R3x1 is zero vector.
5

---

Published as a conference paper at ICLR 2025
Rotate
Construct Frame
Spherical Coordinate
Transiate
Equivariant MoliculaaFragment
Equivariant MolcculsFragment Local Frame
Local Fraire
Frame
Point.
Rottat
Thair late
Point
Figure 3: Illustraions of equivariant local frames. Left: If the molecule is translated and rotated, the molecule and fragment local frames transformed accordingly. Right: Local frames constructed using the first three non-collinear points, and the spherical coordinates are obtained under the local frame. More details shown in Section 3.3.2andd and Section 3.3.3
Then, we can use the chain rule of coordinate frame transformations to obtain the homogeneous transformation between molecule and fragment local frames. Formally,
T g- = T
m--w T -
Tg-w
RT
m-
PT Tmn--w +
Ro-w
t f--
0
(3)
R wm
R -
R
ww (tg-
tm -w
From the homogeneous transformation matrix Tg-om: we can extract the rotation matrix Ry-+m which will be used to obtain invariant representations as described in Section 3.3.3Jmm and the translation vector b--m such that
Rg-sm = Rd
-w Rg-sws fg->m
Rd vw
( fm--w).
For each fragment in the fragment dictionary, apart from atom types and atom local coordinates, also need to save the displacement (g-) between the origin of the fragment local frame and the fragment center, which can be calculated via
t (g)) = V tg-m-
(5)
where c(G) denotes the center of any fragment G. and Vm represents the coordinates of the fragment center under the molecule local frame. (g))(()) will be used when converting local coordinates
from fragment local frame back to the world frame, which is described in Section 3.3.3
3.3.3 -333)-INAARIANT FRAGMENT LOCAL REPRESENTATIONS
After establishing the frames, we use a function f(-) to convert the coordinates of each fragment center spherical coordinates d, f, under the molecule frame (x, y, x). Specifically, for each fragment center & with coordinate Vú,, the corresponding spherical coordinate is
de |ve, Ur, 1l,, Oz.- arccos ((vi, - -U;,) z/de,),
(6)
de. = atan2 ((v,. ve,) y, (vg, ve,) x).
The spherical coordinates show the relative position of each fragment under the molecule frame m. As shown in Figure 3 if the molecular coordinates are rotated by a matrix R and translated by vector the transformed spherical coordinates remain the same, so the spherical coordinates are SE(3)- invariant. Compared to Cartesian coordinates, spherical coordinate values are bounded in smaller region, namely, range [0, =)/(0, 2x). Given same numerical constraints, spherical coordinates require smaller vocabulary size, and given the same vocabulary size, spherical coordinates present less information loss. This makes spherical coordinates advantageous in discretized representations and thus easier be modeled by LMs. Experiments also show the superiority of invariant spherical coordinates over invariant Cartesian coordinates, as detailed in Appendix [A.4
Apart from the spherical coordinates of the fragment center point under the molecule local frame, we also need the orientation the fragment local frame with respect to the molecule local frame. One option is to directly use the rotation matrix Ry-m derived above. However, the rotation

---

Published as a conference paper at ICLR 2025
matrix representation has redundant information since 3D rotation in Euclidean space only has three degrees of freedom, and it can also unnecessarily increase the context length for LMs. To have more compact rotation representation, and fragment local frames to obtain a rotation vector, which indicates the rotation axis and angle. Specifically, we first calculate the rotation matrix Rg-rm in g(=;)) "), as described in Section 3.3.2 then we can obtain the rotation angle ab and rotation axis a = (az, ay as) from the rotation matrix. Next, we can calculate rotation vector m (my My, m2) via m v'a Detailed derivation of vi and can be found in Appendix B
function g(:,.) that takes in both the molecule
Conversely, given the spherical coordinates of the fragment centers under the molecule local frame and the rotation vector m between the fragment and molecule local frames, we can derive the atom coordinates in the world frame. Specifically, we first convert spherical coordinates to Cartesian coordinates V and transform the rotation vector back to the rotation matrix Rg Rg-m- Then, we can coordinate transformation to convert atom coordinates from the fragment local frame to the world frame. Formally,
f--m = Vm ((C) 19))a()): V = R gm + 4- V = Vm R + lm mm-ww-
where (i) denotes atom coordinates under certain coordinate frame and (g))(()) can be retrieved from the fragment dictionary built in Section [33.2] As for Rm-w and fm-ww they are associated with the reference molecule for given protein pocket when we construct the molecule local frame, and we can still use them when we generate new molecules for the same protein pocket.
Formally, with the (3) eqquivriiant local frames constructed in Section 3.3.2 we have the following properties of local representations built in this section.
Lemma 3.3. Let M (V, be a 3D molecular graph with node type matrix Z and node coordinate matrix V. Let m be equivariant local frames of M built based on the first three non-collinear fragment centers in L(M) and g be equivariant local frames of any fragment G3 built based on the first three non-collinear atoms in L(G.). f()) is our function that maps 3D coordinate matrix V of M spherical representations S under the molecule local frame m. g(:-) is the function that maps the molecule local frame in and fragment local frame to rolation vectors m. Then for any 31) transformation
S F3)), we have [(V) [[V(V) and g(m,g) .)) = (m, g)). Given spherical representations S = f(V) and rotation vectors m = g(m, g). there exist a transformation T E SE)3), such that f 1- (S) rVV and g (m) = r(m, g).
Lemma 3.3 shows that our spherical and rotation vector representations are SE(3) -invarnnvrii and we reconstruct original molecules from those representations, differing only by a SE(3) transforma- tion. Detailed proof is provided in Appendix C.2
3.3.4 FRAG2SEQ: FRAGMENT AND GEOMETRY AWARE TOKENIZATION
With ordering that reduces molecule structures to ID sequences and and rotation representations, both ensuring minimum 3D information loss, we develop a reversible transformation from 3D molecular fragments to ID sequences. Figure 2 shows an overview of our method, which we term as Frag2Seq. Specifically, given molecule M with k fragments, we concatenate the fragment-pe possition vector [s,, d,, 6,, Øi, Miii Myi m,s] of every fragment Gj in M into a ID sequence by its canonical order, l,,... la, where 8; is the canonical SMILES string of G, To define its properties, we formulate Frag2Seq and our major theoretical derivations below.
spherical
and M2 (V2, Z2) be Iwo 3D molecular graphs. Let L(M) be the canonical order for M and f(-) and g(---) be the functions following Lemma 3.3 Let the fragment-position vector for a molecule fragment G, be x; [s, d2, 6s, Φs, Mas. Myss mzd. where the vector elements are derived from f(-) and g(.--) -) as predefined. For the fragments )im M,, we construct canonical order L(M) for the k fragments, l..... ,L,, specifically by the order of a fragment's 's first-ranked atom. We define Frag2Seq M - U, which maps a molecule M E M. to its sequence representation U E U. the set of all possible sequence representations, as
Theorem 3.4. [Bijective Mapping] Following Def. 3.1 let M, -Vi,
xtt)) ); where concat(-) concatenates elements as a sequence. Then Frag2Seq()) .) is a ffncictiov and the following equivalence holds: Frag2Seq(M11) Frag2Seq(Ma) + Mi Sa M2,
(rag2Seq(M) concat(a?,
7

---

Published as a conference paper at ICLR 2025
PDB 4m7t
Win: -10.0 QED: 0.47
Vina: -1.0 QED: 0.70
Vina: 10.6 QED: 0.83
Vina: -11.0 QED: 0.62
Vine: -7.8 QED 0.e3
Figure 4: Visualization of our generated 3D molecules that have higher binding affinity than the reference molecule in the test set. A higher QED indicates more niug-likeees, and lower Vina score indicates higher binding affinity.
where M1 3DD M2 denotes M1 and M2 are 3D isomorphic. we allow rounding up spherical coordinate and rotation vector values Z b decimal places, then the surjectivity and equivalence still hold, only molecules M_ and M2 are (100~)/2)-coonstrinn] 3D isomorphic.
Theorem estaalshlss guarantees that
uniquely construct 1D sequence given a 3D molecule using Frag2Seq, and uniquely reconstruct a 3D molecule given a sequence output of Frag2Seq. Furthermore, two sequence outputs from Frag2Seq are identical if and only if the two corresponding molecules are 3D isomorphic. This enables sequential tokenization of 3D molecules while preserving structural completeness and geometric invariance. Due the necessity of discreteness in tokenization for LMs, in reality, numerical values need to be discretized before concatenation. In practice, we round up numerical values certain decimal places. In this case, Theorem also extends guarantees for the practical use of Frag2Seq. If we allow round-up error below 110 T02 for coordinates when distinguishing 3D isomorphism, all properties still hold. This implies that the practical Frag2Seq implementation retains near-complete geometric information and invariance, with numerical precision of c S [10 11/2.
With discreteness incorporated, can collect finite vocabulary covering all accessible fragment samples to enable tokenization for LMs. Specifically, we use vocabularies consisting of fragment type tokens such O, O=C, elecceel, The numerical tokens range from the smallest to the largest distance, angle, and rotation values with restricted precision of 2 or decimal places.
and numerical tokens such as 8.126, 0.05 1.7022.
3.4 CONDITIONAL TRAINING AND GENERATION
After we obtain the sequence representation of 3D molecules U = {[u1,". Un ). we need to model the distribution of such 3D geometry-aware sequences. We adopt GPT (Radford et al.) 20188) as the base model to learn the distribution. To incorporate the protein pocket condition, we use pre-trained inverse folding model ESM-IFF (Hsu et al. 2022) to obtain node embeddings of the protein backbone. Then, we add cross-and aatenntttnn between protein node embeddings and ligand token embeddings after the multi-head self-attention in each attention block, in which queries are from ligand token embedding and keys and values are from protein node embedding, For training, we use the standard next-token prediction cross-entropy loss to maximize the following likelihood:
L(U:P) Σ log po (u; (u 011 P))
To generate molecules conditioned a protein pocket, we first need to sample initial fragment tokens from the first-token distribution in the training set. Then, we can generate the following tokens autoregressively by sampling from the conditional distribution pe (u./u-11,. , M1; P) and stop the generation when the maximum length is reached or the ending token is sampled.
EXPERIMENTS
We empirically show the effectiveness our method in generating ligands for protein pockets. In Section 4.1 we describe the experimental setup, including dataset, baselines, and evaluation metrics. Then, in Section 4.2 we present our results about binding affinity, chemical properties, and efficiency. In Section 4.3 show more results about structural analysis. Due to the limited space, we conduct ablation studies on several design choices Appendix A.4, and more results about efficiency in Appendix A.5.
8

---

Published as a conference paper at ICLR 2025
Table I: Summary of molecular properties for generated molecules by our method and other baselines.
denotes the property results taken from the DiftSBDD paper. f denotes that we re-evaluate the
properties of generated molecules of those basclines using the DifiSBDD evaluation code for fair
comparison. The best results are annotated in bold.
Methods
Vina Score (1)
High Affinity (1)
QED (t)
(A)
Lipinski (*)
Diversity (t)
Time (s. L)
Test
-6.77 2.32
0.176 0.20
0.728 =: 0.14
4,340 1.14
3D-SBDD=
-5.88 1.91
0.364 0.31
0.502 0.11
0.675 =: 0.14
4.787 =: 0.51
0.742 = 0.00
15086 9851
-T.088 2.80
0.515
0.31
0.572 + 0.16
0.752 0.12
4.938 + 0.22
0.735 +015
2527.3 + 1426.8
GraphBP
0.183
0.502
0.307
4,883
0.844
138.5
TargeiDifF>
-7,318 = 2.47
0.581 0.31
0,483 1: 0.20
0.584 1: 0.13
4.594
0.83
0.718 a: 0.09
3428
DecompDidi
6.607 2.11
0.423 0.25
01496 + 0.21
0.659 + 0.14
4.493 + 1.02
0.722 +-300
~ 6189
DinSBDD*
7.177
0.499 0.30
0.556
0.729 0.12
4.742
0.718
629.9
FLAGGi
6.389 3.24
0.478 + 0:34
0.487 0..19
0.702 +0155
4,638 0.74
0.701 +.114
1289.1 378.0
DrogGPSt
-6,008 2.38
0.121 0:21
0.167 0.22
0.628 0.15
0.99
0.738 0.10
1007 8 154.1
Ling
0.625
0.36
0,289
0.658 0.08
3.121
0.480 0.12
1512.8
FragZSeq
-7.366 1.96
0.653 1: 0.33
0.645 0.15
0.612:0..
4.989 = 0.11
0.711 = 0.07
48.8 =: 111.
4.1 SETUP
Dataset. Following Schneuing et al. (2022), we use the CrossDocked dataset (Francoeur et al. 2020
that is further curated by previous work (Luo al. 202T). More details about the dataset is provided in Appendix A.I
Baselines. We compare with recent generation methods in structure-based drug design. 3D- SBDD (Luo et al. 2021), Pocket2MMol (Peng et al. 2022). and GraphBP (Liu et al. 2022a) generate atoms in autoregressive scheme. TargetDiff (Guan et al. 2022), DecompDiff (Guan et al. 2023), and DiftSBDD (Schneuing et al. 2022) are diffusion-
methods that generate all atoms in one shot. FLAG (Zhang al. (022) and DrugGPS (Zhang & Liu 2023) generate molecules motif-by- motif. Lingo (Feng et al.
2024) uses language model to generate fragment-basee SMILES
and then predict coordinates. The language model is pre-trained on 12 million 3D molecules using a denoising strategy.
Evaluation Metries. We adopt commonly-used metrics used in previous work (Schneuing et al.
2022 Peng et al. 2022 to evaluate the quality of our generated molecules: (1) Vina Score estimates the binding affnity between generated molecules and given protein pockets; (2) High Affinity measures the percentage of generated molecules that have higher binding affinity than the reference molecule for certain protein; (3) QED a measure used to assess the drug-likeness of a molecule based on its molecular properties; (4) SA estimates how easy it would be to synthesize given chemical compound: (5) Lipinski measures how well a molecule satisfies the Lipinski's 's rule of five (Lipinski et al. 2012), which evaluates the frug-likeness, of molecule; (6) Diversity measures the average pairwise diversity (calculated as 1 - Tanimoto similarity) of generated molecules for a binding pocket; (7) Time is the average time cost to generate 100 molecules for a protein pocket in the test set. All the Vina scores are calculated by QVina (Alhossary et al. 2015), , and the chemical properties are calculated by RDKit (Bento et al. 2020
4.2 RESULTS
We show the results of the above metrics for our method and baselines in Table T The results show that our method can achieve better or competitive performance with other baseline methods. Specifically, our method achieves the best result on QED and Lipinski, which indicates that our generated molecules possess better drug characteristics. Additionally, our method obtains the best Vina score and high affinity metric, which shows that our model captures more accurate interactions between ligands and protein pockets to improve the binding affinity. Moreover, our method has much better sampling efficiency than the baselines due to our simple generation pipeline and fragment- based generation strategy.
In Figure El we provide several examples of our generated 3D molecules for a certain protein pocket (PDB id 4m7t). The reference molecule is provided in the test set, and our generated molecules exhibit higher binding affinity than the reference molecule. These examples can validate that our method has the capability to capture the complex interaction between proteins and ligands in order

---

Published as a conference paper at ICLR 2025
30-5800 5D: 0.425
TargetDiff FD: 0.225
Frag25ie 0.158
Pocket2Meol
FLAG
LingoRDMal JSD: 0.385
Targsti
3D 5200
"LAG
returence
LingddMMee
Negttion
2n
T4
Distance bond
Distance carbon carbon (A)
Figure 5: Comparison of the distribution of carbon-carbon bond distance. Jensen-Shannon divergence (ISD) between reference molecules and generated molecules reported.
generate novel molecules that have higher binding affinity while retaining similar or even better drug-likeness properties than the reference molecule.
4.3 STRUCTURAL ANALYSIS
Figure we plot the empirical distribution of carbon-carbon bond distances of generated molecules and reference molecules. The reference distribution (gray) has two modes, but most methods can only capture one mode due to mode collapse. Even though TargetDiff exhibits two modes in the distribution, it still suffers from the over-smoothness issue. Comparatively. our method can better capture those two modes in the reference distribution. Furthermore, we calculate Jensen-Shannon divergence (JSD) (Lin 1991) between the bond distance distribution of reference and generated molecules for each method to quantitatively evaluate the distribution fitting capability. We can observe that our method outperforms other methods by clear margin.
To further evaluate the structures of generated molecules, we calculate the Kullback-Leibler (KL) divergence of bond angles and dihedral angles between generated molecules and reference molecules in the test set. The results in Table 2 show that our method achieves better KL divergence than other representative baselines in most cases, which demonstrates that Frag2Seq can well capture the structural distribution of the data and generate more valid substructures.
Table The KL divergence of the distribution the bond and dihedral angles between generated molecules and reference molecules. The lower letters indicate the atoms in the aromatic rings. * denotes numbers taken from the original paper due to the absence of those dihedral angles in regenerated structures.
Angles
LIAN
3D-SBDD
Pocker2Mol
TargetDiff
FLAG
Frag2Seq
CCC
1.266
0.465
0.770
0.507
0.497
0.263
CCO
1.475
0.618
1.104
0.713
0.768
0.306
NCC
1.312
0.619
0.754
0.585
0.427
0.578
CCCC
0.197
0.250
0.215
0.170
0.164
0.113
cece
0.786
1.044
4.49*
0.229
0.516
0.041
CCCO
0.206
0.263
0.245
0.191
0.180
0.198
Ccce
0.725
0.939
2.85'
0.397
0.311
0.109
CC=CC
0.526
0.655
0.296
0.301
0.345
0.133
CONCLUSIONS, BROADER IMPACTS, AND LIMITATIONS
In this work, we develop a method convert molecules into fragment-based S3F se- quences which can be naturally processed by language models. We use protein inverse folding model to obtain protein embeddings and integrate them into the language model via cross-attention. Our experiments show the great potential of using language models in structure-based drug design, which can potentially accelerate drug discovery by enhancing the efficiency and accuracy of identi- fying viable drug candidates. However, method also has limitations. The chemical bonds not directly generated by the language model and need to be inferred by OpenBabel (O Boyle et al. 2011), and real number tokenization requires quantization, leading some accuracy loss.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported partially by National Science Foundation grant IIS-2243850 and National Institutes of Health grant U01AG070112 12 (to S.J.), and by the Molecule Maker Lab Institute (to H.J.): an AI research institute program supported by NSF under award No. 2019897. The views and conclusions contained herein those of the authors and should be interpreted as necessarily representing the official policies, either expressed or implied, of the U.S. Government. The U.S. Gov- emmment authorized to reproduce and distribute reprints for govermmental purposes notwithstanding any copyright annotation therein.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. GPT-4 technical report. arXiv preprint arXiv:2303.08771 2023.
Amr Alhossary, Stephanus Daniel Handoko, Yuguang Mu, and Chee-Keong Kwoh. Fast, accurate, and reliable molecular docking with quickvina Bioinformatics, 31(13):2214-2216 2015.
Amy C Anderson. The process of structure-based drug design. Chemistry & biology, 10(9):787-797, 2003.
Viraj Bagal, Rishal Aggarwal, PK Vinod, and U Deva Priyakumar. MolGPT: molecular generation using a cransformer-dcodde model. Journal of Chemical Information and Modeling, 62(9): 2064-2076, 2021.
Jürgen Bajorath. Chemical language models for molecular design. Molecular Informatics, 43(1): e202300288, 2024.
A Patrícia Bento, Anne Hersey, Eloy Félix, Greg Landrum, Anna Gaulton, Francis Atkinson, Louisa Bellis, Marleen De Veij, and Andrew R Leach. An open source chemical structure curation pipeline using rdkit. Journal of Cheminformatics, 12:1-16, 2020.
Andrew E Blanchard, Debsindhu Bhowmik, Zachary Fox, John Gounley, Jens Glaser, Belinda Akpa, and Stephan Irle. Adaptive language model training for molecular design. Journal of Cheminformatics, 15(1):1-12, 2023.
Andres M Bran and Philippe Schwaller. Transformers and large language models for chemistry and drug discovery. arXiv preprint arXiv:2310.06088 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1901, 2020.
He Cao, Zijing Liu, Xingyu Lu, Yuan Yao, and Yu Li. InstructMol: Multi-modal integration for build- ing a versatile and reliable molecular assistant drug discovery. arXiv preprint arXiv:2311.16208, 2023.
Yangyang Chen, Zixu Wang, Xiangxiang Zeng. Yayang Li, Pengyong Li, Xiucai Ye, and Tetsuya Sakurai. Molecular language models: Rnns or transformer? Briefings in Functional Genomics, pp. elad012, 2023a.
Yuhan Chen, Nuwa Xi, Yanrui Du, Haochun Wang, Chen Jianyu, Sendong Zhao, and Bing Qin. From artificially real to real: Leveraging pseudo data from large language models for low-resource molecule discovery. arXiv preprint arXiv:2309.05200 05230090052.,.. 2023b.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. PaLM: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-115, 2023.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint / 180000400, 2018.
11

---

Published as a conference paper at ICLR 2025
Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. arXiv preprint arXiv:2204118817, 2022.
Yin Fang, Ningyu Zhang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Molecular language model as multi-task generator. arXiv preprint arXiv: 1..300111555 1259,, 2023.
Wei Feng, Lvwei Wang, Zaiyun Lin, Yanhao Zhu, Han Wang, Jianqiang Dong, Rong Bai, Huting Wang, Jielong Zhou, Wei Peng, et al. Generation of 3d molecules in pockets via a language model. Nature Machine Intelligence, pp. 1-12, 2024.
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B Iovanisci, Ian Snyder, and David R Koes. Three diree-dimensional convolutional neural networks and cross-docked data set for structure-based drug design. Journal of chemical information and modeling, 60(9):4200-42) 4215, 2020.
Nathan C Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Gomez-Bombarelli, Con- nor Coley, and Vijay Gadepally. Neural scaling of deep chemical models. Nature Machine Intelligence, 5(11):1297-1305 2023.
Cong Fu, Xuan Zhang, Huixin Zhang, Hongyi Ling, Shenglong Xu, and Shuiwang Ji. Lattice convolutional networks for learning ground states of quantum many-body systems. arXiv preprint Stffv:2206.037777 2022.
Cong Fu, Jacob Helwig, and Shuiwang Ji. supi-supe learning for fighfiielity fluid flow reconstruction. In Learning on Graphs Conference, pp. 36-1. PMLR, 2024a.
Cong Fu, Keqiang Yan, Limei Wang, Wing Yee Au, Michael Curtis McThrrow, Tao Komikado, Koji Maruhashi, Kanji Uchino, Xiaoning Qian, and Shuiwang Ji. A latent diffusion model for protein structure generation. In Learning on Graphs Conference, pp. 29-1. PMLR, 2024b.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces, arXiv preprint we.iiv:211..000 2021.
Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng. and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. In The Eleventh International Conference on Learning Representations, 2022.
Jiaqi Guan, Xiangxin Zhou, Yuwei Yang. Yu Bao, Jian Peng. Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu. Decompdiff: Diffusion models with decomposed priors for structure-based drug design. Proceedings of Machine Learning Research, 202: 1827-11628. 2023.
Shurui Gui, Xiner Li, and Shuiwang Ji. Discovering physics laws of dynamical systems via invariant function learning. arXiv preprint arXiv:2502.0444: 2025.
Suhail Haroon, CA Hafsath, and AS Jereesh. Generative pre-trained transformer (GPT) based model with relative attention for de novo drug design. Computational Biology and Chemistry, 106:107911, 2023.
Jacob Helwig, Xuan Zhang, Cong Fu, Jerry Kurtin, Stephan Wojtowytsch, and Shuiwang Ji. Group equivariant fourier neural operators for partial differential equations. arXiv preprint arXiv:2306.0m 05697, 2023.
Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. In International conference on machine learning, pp. 8946-8970. PMLR, 2022.
Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. Trustlim: Trustworthiness in large language models. arXiv preprint 2tt.::240100556... 2024a.
Zhilin Huang, Ling Yang, Xiangxin Zhou, Zhilong Zhang, Wentao Zhang, Xiawu Zheng, Jie Chen, Yu Wang, CUI Bin, and Wenming Yang. Protein-ligand interaction prior for binding- 3d molecule diffusion models. In The Twelfth International Conference on Learning Representations, 2024b.
12

---

Published as a conference paper at ICLR 2025
Adam Izdebski, Ewelina Weglarz-Tomezakk Ewa Szczurek, and Jakub M Tomezak. De novo drug design with joint transformers. arXiv preprint arXiv:2310.02066, 2023.
Nikita Janakarajan, Tim Erdmann, Sarath Swaminathan, Teodoro Laino, and Jannis Born. Language models in molecular discovery. arXiv preprint arXiv:2309, 2023,
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometrie vector perceptrons. International Conference on Learning Representations, 2020.
Sambhav Khurana, Xiner Li, Shurui Gui, and Shuiwang Ji. A hierarchical language model for interpretable graph reasoning. arXiv preprint arXiv:2410.22372, 2024.
Dmitrii Kochkov, Tobias Pfaff, Alvaro Sanchez-Gonzalez, Peter Battaglia, and Bryan K Clark. Learn- ing ground states of quantum hamiltonians with graph networks. arXiv preprint arXiv:21 2021.
Mario Krenn, Florian Häse, A Nigam, Pascal Friederich, and Alán Aspuru- Guzik. Selfies: robust representation of semantically constrained graphs with an example application in chemistry. arXiv preprint arXiv:1905.1374 1(3), 2019.
Gregory W Kyro, Anton Morgunov, Rafael I Brent, and Victor S Batista. Chemspaceal: An efficient active learning methodology applied to protein-specific molecular generation. ArXiv, 2023.
Jiatong Li, Yunqing Liu, Wenqi Fan, Xiao Xiao-Yong Wei, Hui Liu, Jiliang Tang, and Qing Li. Empowering molecule discovery for molecule-caption translation with large language models: A chatgpt perspective. arXiv preprini arXiv:2306.066 2306.06616 2023a.
Xiner Li, Shurui Gui, Youzhi Luo, and Shuiwang Ji. Graph structure and feature extrapolation for out-of-distribution generalization. arXiv preprint arXiv:2306.088 08076, 2023b.
Xiner Li, Limei Wang, Youzhi Luo, Carl Edwards, Shurui Gui, Yuchao Lin, Heng Ji, and Shuiwang Ji. Geometry informed tokenization of molccules for language model generation. arXiv preprint arXiv:2408. 10120, 2024a.
Xiner Li, Yulai Zhao, Chenyu Wang. Gabriele Scalia, Gokcen Eraslan, Surag Nair, Tommaso Biancalani, Shuiwang Ji, Aviv Regev, Sergey Levine, et al. Derivative-free guidance continuous and discrete diffusion models with soft value-based decoding. arXiv preprint arXiv: wrXiv:2408.082522 2024b.
Yuesen Li, Chengyi Gao, Xin Song, Xiangyu Wang, Yungang Xu, and Suxia Han. DrugGPT: A GPT- based strategy for designing potential ligands targeting specific proteins. bioRxiv, pp. 2023-06, 2023c.
Youwei Liang, Ruiyi Zhang, Li Zhang, and Pengtao Xie. DrugChat: towards enabling ChatGPT-like capabilities on drug molecule graphs. arXiv preprint wxxiv:330.033007 2023.
Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Transactions on Information theory, 37(1):145-1511 1991.
Christopher A Lipinski, Franco Lombardo, Beryl W Dominy, and Paul J Feeney. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced drug delivery reviews, 64:4-17, 2012 2012.
Meng Liu, Cong Fu, Xuan Zhang, Limei Wang, Yaochen Xie, Hao Yuan, Youzhi Luo, Zhao Xu, Shenglong Xu, and Shuiwang Ji. Fast quantum property prediction via deeper 2d and 3d graph networks. arXiv preprint arXiv:2106.0855 2021a.
Meng Liu, Youzhi Luo, Limei Wang. Yaochen Xie, Hao Yuan, Shurui Gui, Haiyang Yu, Zhao Xu, Jingtun Zhang, Yi Liu, et al. Dig: A turnkey library for diving into graph deep learning research. The Journal of Machine Learning Research, 22(1):00733-108) 2021b.
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. In International Conference on Machine Learning. pp. 19992-11924. PMLR, 2022a.
13

---

Published as a conference paper at ICLR 2025
Shengchao Liu, Weili Nie, Chengpeng Wang. Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar. Multi-modal molecule structure- text model for retrieval and editing. Nature Machine Intelligence, 5(12): 2023a.
Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3D molecular graphs. In International Conference on Learning Representa- tions, 2022b.
Zequn Liu, Wei Zhang. Yingce Xia, Lijun Wu, Shufang Xie, Tao Qin, Ming Zhang, and Tie- Yan Liu. Molxpt: Wrapping molecules with text for generative pre-training. arXiv preprint mrXiv:2305.10680 2023b.
Hya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 1rXiv: 2017.
Renqian Luo, Liai Sun, Yingee Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie- Yan Liu. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, bbb6)4bb 2022.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure based drug design. Advances in Neural Information Processing Systems, 14:6229-62399 2021.
Paul D Lyne. Structure-based virtual screening: overview. Drug discovery today, 7(20): 1047-1050 2002.
Jiashun Mao, Jianmin Wang, Amir Zeb, Kwang-Hww Cho, Haiyan Jin, Jongwan Kim, Onju Lee, Yunyun Wang, and Kyoung Tai No. Deep molecular generative model based on variant transformer for antiviral drug design. Available at SSRN 43458/ 1, 2023a.
Jiashun Mao, Jianmin Wang, Amir Zeb, Kwang-HHe Hwi Cho, Haiyan Jin, Jongwan Kim, Onju Lee, Yunyun Wang, and Kyoung Tai No. Transformer-baset molecular generative model for antiviral drug design. Journal of Chemical Information and Modeling, 2023b.
Eyal Mazuz, Guy Shtar, Bracha Shapira, and Lior Rokach. Molecule generation using transformers and policy gradient reinforcement learning. Scientific Reports, 13(1):8799, 2023.
Brendan D McKay and Adolfo Piperno. Practical graph isomorphism, ii Journal of symbolic computation, 60:94-112, 2014.
Brendan D McKay et al. Practical graph isomorphism. 1981.
Noel M O' Boyle, Michael Banck, Craig A James, Chris Morley, Tim Vandermeersch, and Geoffrey R Hutchison. Open babel: An open chemical toolbox. Journal of cheminformatics, 3:1-14, 2011.
Noel M O' O'Boyle. Towards a universal smiles representation-a standard method to generate canonical smiles based on the inchi. Journal of cheminformatics, 4:1-14, 2012.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, peggorerformance deep learning library. Advances in neural information processing systems, 32, 2019.
Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingee Xia, and Rui Yan. Biot5: Enriching cross cross-modal integration in biology with chemical knowledge and natural language associations. arXiv preprint 2xXiv:3310002211 2023.
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. In International Conference on Machine Learning, pp. 17644-17655. PMLR, 2022.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, al. Improving language understanding by generative pre-training. 2018.
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3d molecules conditional on receptor binding sites with deep generative models. Chemical science, 1309):771-2713, 2022.
14

---

Published as a conference paper at ICLR 2025
Alvaro Sanchez- Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International conference machine learning. pp. 8459-8468. PMLR, 2020.
Arne Schncuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, et al. Structure-based drug design with equivariant diffusion models. arXiv preprint arXhv:2210.13699 2022.
Brian K Shoichet. Virtual screening of chemical libraries. Nature, 432(7011):862 865, 2004.
Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology, 35(11):1026-1028 2017.
Hugo Touvron, Thibaut Lavril, Gautier Izacard. Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13911, 2023.
Emma P Tysinger, Brajesh K Rai, and Anton V Sinitskiy. Can quickly learn "translate" bioactive molecules with transformer models? Journal of Chemical Information and Modeling, 63 (6):1734-1744, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. ComENet: Towards complete and efficient message passing for 3d molecular graphs. In Advances in Neural Information Processing Systems, 2022. URL WRRLhttps:: hittps::/openreviiee
net mommff d=mCZMgeWSFJ
Zifeng Wang. Zichen Wang. Balasubramaniam Srinivasan, Vassilis N Ioannidis, Huzefa Rangwala, and Rishita Anubhai. Biobridge: Bridging biomedical foundation models via knowledge graph. arXiv preprint a arXiv:2310.03320, 2023.
Ziqi Wang, Hanlin Zhang, Xiner Li, Kuan-Hac Huang, Chi Han, Shuiwang Ji, Sham M Kakade, Hao Peng, and Heng Ji. Eliminating position bias of language models: A mechanistic approach. arXiv preprint ar arxiv:2407.0 000.0/111 100, 2024.
David Weininger. SMILES, chemical language and information system. 1. introduction to method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36, 1988.
David Weininger, Arthur Weininger, and Joseph L Weininger. Smiles. 2. algorithm for generation of unique smiles notation. Journal of chemical information and computer sciences, 29(2):771101, 1989.
Juan-Ni Wu, Tong Wang, Yue Chen, Li-Juan Tang, Hai-Long Wu, and Ru-Qin Yu. Fragment-based (-smiles for de novo molecular generation. arXiv preprint arXiv:2301.018 001029,,,,,,,,,, 2023.
Jun Xia, Yanqiao Zhu. Yuanqi Du, Liu, and SZ Li. A systematic survey of chemical pre-trained models. UCAI, 2023.
Yingce Xia, Kehan Wu, Pan Deng, Renhe Liu, Yuan Zhang, Han Guo, Yumeng Cui, Qizhi Pei, Lijun Wu, Shufang Xie, et al. Target-aware molecule generation for drug design using chemical language model. bioRxiv, pp. 2024-01, 2024.
Tong Xie, Yuwei Wan, Wei Huang, Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, Chunyu Kit, Clara Grazian, Wenjie Zhang, et al. DARWIN series: Domain specific large language models for natural science. arXiv preprint arXiv:2308. 13565, 2023.
Zhongyin Xu, Xiujuan Lei, Mei Ma, and Yi Pan. Molecular generation and optimization of molecular properties using transformer model. Big Data Mining and Analytics, 7(1):12--155) 2023.
Keqiang Yan, Cong Fu, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. Complete and efficient graph transformers for crystal material property prediction. arXiv preprint a4X.v:2403.1 1857, 2024.
15

---

Published as a conference paper at ICLR 2025
Keqiang Yan, Xiner Li, Hongyi Ling, Kenna Ashen, Carl Edwards, Raymundo Arróyave, Marinka Zitnik, Heng Ji, Xiaofeng Qian, Xiaoning Qian, et al. Invariant tokenization of crystalline materials for language model enabled generation. Advances in Neural Information Processing Systems, 37: 125050-125072, 2025.
Yasuhiro Yoshikai, Tadahaya Mizuno, Shumpei Nemoto, and Hiroyuki Kusuhara. Difficulty in learning chirality for transformer fed with smiles. arXiv preprint arXiv:2303.1159) 2023.
Qiang Zhang, Keyang Ding, Tianwen Lyv, Xinda Wang. Qingyu Yin, Yiwen Zhang, Jing Yu, Yuhao Wang, Xiaotong Li, Zhuoyi Xiang, et al. Scientific large language models: A survey on biological & chemical domains. arXiv preprint arXiv:2401. 14656, 2024a.
Weitong Zhang, Xiaoyun Wang, Weili Nie, Joe Eaton, Brad Rees, and Quanquan Gu. MoleculeGPT: Instruction following large language models for molecular property prediction. In NeurlPS 2023 Workshop on New Frontiers of Al for Drug Discovery and Development, 2023a.
Xuan Zhang, Limei Wang. Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, Keir Adams, Maurice Weiler, Xiner Li, Tianfan Fu, Yucheng Wang. Haiyang Yu, YuQing Xie, Xiang Fu, Alex Strasser, Shenglong Xu, Yi Liu, Yuanqi Du, Alexandra Saxton, Hongyi Ling, Hannah Lawrence, Hannes Stärk, Shurui Gui, Carl Edwards, Nicholas Gao, Adriana Ladera, Tailin Wu, Elyssa F. Hofgard, Aria Mansouri Tehrani, Rui Wang. Ameya Daigavane, Montgomery Bohde, Jerry Kurtin, Qian Huang, Tuong Phung. Minkai Xu. Chaitanya K. Aspuru-Guzik, Joshi, Simon V. Mathis. Kamyar Azizzadenesheli, Ada Fang, Alán Aspuru Guzik, Erik Bekkers, Michael Bronstein, Marinka Zitnik, Anima Anandkumar, Stefano Ermon, Pietro Liò, Rose Yu, Stephan Günnemann, Jure Leskovec, Heng Ji, Jimeng Sun, Regina Barzilay, Tommi Jaakkola, Connor W. Coley, Xiaoning Qian, Xiaofeng Qian, Tess Smidt, and Shuiwang Ji. Artificial intelligence for science quantum, atomistic, and continuum systems, 2023b.
Xuan Zhang. Jacob Helwig, Yuchao Lin, Yaochen Xie, Cong Fu, Stephan Wojtowytsch, and Shuiwang Ji. Sinenet: Learning temporal dynamics in time-dependent partial differential equations. arXiv preprint arv:v:2403. 19507, 2024b.
Zaixi Zhang and Qi Liu. Learning subpocket prototypes for generalizable structure- atructure-based drug design. In International Conference on Machine Learning, pp.
PMLR, 2023.
Zaixi Zhang, Yaosen Min, Shuxin Zheng, and Qi Liu. Molecule generation for target protein binding with structural motifs. The Eleventh International Conference on Learning Representations, 2022.
Xiangxin Zhou, Xiwei Cheng, Yuwei Yang, Yu Bao, Liang Wang. and Quanquan Gu. Decompopt: Controllable and decomposed diffusion models for structure-based molecular optimization. arXiv preprint arXiv:2403. 13829, 2024.
16

---

Published as a conference paper at ICLR 2025
A EXPERIMENTAL DETAILS AND ADDITIONAL RESULTS
A.1 DATASET DETAILS
Following previous work in SBDD (Schneuing et al. 2022 Luo et al. 2021 Peng et al., 2022 Guan et al., 2022 2023), we use the CrossDocked2020 dataset train and test our model. The dataset was originally created by [rancoeur et al. (2020) We adopt the version that is further filtered in the previous work (Luo et al., 2021). To increase the data quality, they filter out data points that have binding pose RMSD greater than 1 1 The data split is performed according to 30% sequence identity using MMseqs2 (Steinegger & Söding 2017). Finally. there are 100000 protein-ligand training pairs and 100 proteins in the test set. The dataset curated by Luo et al. (202T) is publicly available at https:://///// / g thub. http://ostttt/tt// www.eeeerraaeeeeeeoeeooa
main/ datt//aaa///// README. md under the MIT license.
A.2 PROTEIN ENCODER DETAILS
We use the pre-trained ESM-IF1 model to obtain protein node embeddings. ESM-IF1 model is also known as model that is used to design protein amino acid sequences given 3D backbone structures. The ESM-IFF model uses a modified Geometric Vector Perceptron (GVP) layers (Ting al. 2020) to extract rotational and translational invariant features, followed by an autoregressive encoder- decoder Transformer (Vaswani al. 2017). The model weights of ESM-IFI are publicly available esm under the MIT license.
that is developed by Hsu et al. (2022). It is an inverse folding
at
A.3 NETWORK AND TRAINING DETAILS
For the language model, we use the GPT-1 architecture (Radford et al. 2018) with 12 layers, 12 attention heads, and a hidden embedding size of 768. Cross-attention is added after self-attention in each layer. The protein embedding size generated by the ESM-IF1 model is 512 and is increased to 768 by an MLP layer before the protein embeddings are fed into the cross-attention. The architecture of the cross-attentions the same as the self-attention, except that the key and value protein embeddings. The initial learning rate is set as 4e, and the batch size is 64. We implement our methods using PyTorch (Paszke et 2019) and we adopt the AdamW optimizer (Loshchilov & Hutter 2017) with 3 = 0.9, 32 = 0.95. We adopt linear warm up and cosine scheduler to adaptively adjust the learning rate. Specifically, during the first 10% of total training tokens, we increase the learning rate from 0 to the initial learning rate. Then, the learning rate decreases to 4e-6 by using a cosine decay scheduler. We use single NVIDIA A100 GPU to train our model. We set the maximum context length as 512 for all the experiments except for the ablation study converting proteins into sequences. In that case, the maximum context length is increased to 850 to accommodate the preappended protein sequences.
from the
A.4 ABLATION STUDY
In this section, we conduct ablations studies on the 3D representation, fragmentation, and protein embedding in order to show how these factors affect the performance of the language model on the SBDD task.
First, we compare the effect of different 3D representations of fragment center under SE(3)- equivariant molecule local frame proposed in Section 3.3.2 We explore the use of SE(3)--ivvriae spherical coordinates used in Section 3.3.3 From
Cartesian coordinates instead of SF the result Table we can see that using spherical coordinates can achieve better binding affinity and chemical properties. By using spherical coordinates, the distances and angles are constrained to smaller region, which makes the language model easier to learn structure correlations.
Then, we show that fragment-based tokenization is crucial to achieving better generation results. Specifically, we try to generate molecules atom-by bo--bby-tom instead of using fragments. In this setting, we do not need to build fragment local frames, and rotation vectors do not exist in converted sequences. Results in Table 4 show that fragment- Tragment-bused generation can achieve much better performance on binding affinity and most likgeeiseness properties, which validates the significance of fragmentation.
17

---

Published as a conference paper at ICLR 2025
Finally, compare different ways to incorporate protein pocket information into the model. Specifi- cally, we convert protein structures to sequences in the same way as molecules. Then, we preappend the tokenized protein sequence to the corresponding ligand sequence and feed it into the language model. Results in T that extracting protein node embeddings and integrating them into the language model via cross-attention more effectively capture the interaction between proteins and ligands.
tokenization small
Table 3: Generation performance between using invariant Cartesian coordinates and spherical coordinates representations of fragment centers.
Methods
Vina Score (4)
QED
SA (1)
Lipinski
Diversity (1)
Sogg2Seq-Invvriane Coordinates
-7.242 2.11
0.636 0.16
0.637 = 0.10
4.974 0.17
0.714 0.09
Trag2Seq-Spherieee Coordinates
-7.366 1.96
0.645 + 0.15
0.642 + 0.11
4.989 + 0.11
0.711 0.07
Table 4: Generation performance between using atom-by-atom and fragment -orronnnt-b--rrmenntt manners.
Methods
Vina Score (1)
QED (t)
SA (t)
Lipinski (1)
Diversity (t)
Frag2Seq-Atone
-6.895 1.42
0.484 0.22
0.683 0.11
4.677 + 0.50
0.737 0.19
Fraggeee-Fragment Fragment
-7.366 + 1.96
0.645 0.15
0.642 + 0.11
4.989 + 0.11
0.711 0.07
Table 5: Generation performance between different ways incorporate protein information.
Methods
Vina Score ())
QED (f)
SA (f)
Lipinski (1)
Diversity (t)
Frag2Seq- Protein Seq
-6.646 + 1.64
0.605 + 0.17
0.691 +: 0.13
4.968 + 0.18
0.493 + 0.39
Frag2Seq-ESMr IEM-IFI
-7.366 1.96
0.645 0.15
0.642 = 0.11
4.989 0.11
0.711 1 0.07
A.5 GENERATION EFFICIENCY ANALYSIS
In this section, compare the generation efficiency of method and other representative baseline methods. We use a single NVIDIA 2080Ti GPU and a batch size of 10. The results in Table/osshow that our method exhibits better sampling efficiency than other autoregressive and diffusion-based models.
Table 6: Generation efficiency comparison among autoregressive methods, diffusion methods, and our fragment-based LM method.
Methods
Parameters
Memory
Sample/second
3D-SBDD
1.2M
3.4GB
0.005
Pocket2Mol
3.7M
1.2GB
0.008
DrugGPS
5.1M
2.5GB
0.73
TafgetDiff
2.8M
1.8GB
0.01
Frag2Seq
134.3M
2.2GB
2.0
A.6 PARAMETER SIZES AND PERFORMANCE COMPARISON
In this section, we compare different parameter sizes of the language models (130M, 70M, 40M). as shown in Table 7 Drug-likeness properties exhibit similar performance across the three models, but there is noticeable degradation in binding energy the model size decreases. We hypothesize this is due to the inherent nature of language models, which require large number of parameters to learn intricate interaction patterns, which is required in learning interactions between proteins and molecules. However, even with large parameter sizes, the language model still offers superior sampling efficiency compared to previous methods, compared in Table
18

---

Published as a conference paper at ICLR 2025
Table 7: Generation performance between different sizes of language models.
Methods
Vina Score (L)
QED (1)
SA (1)
Lipinski (1)
Diversity (f)
Frag2Seq- 130M
-7.366 + 1.96
0.645 0.15
0.642 0.11
4.989 = 0.11 0.11
0.711 0.07
Frag2Seq 70M
-7.255 + 1.83
0.630 + 0.16
0.640 0.11
4.977 + 0.16
0.718 + 0.08
Frag2Seq-40M
-7.228 + 2.09
0.643 + 0.16
0.645 + 0.11
4.982 + 0.14
0.708 + 0.07
A.7 COMPARISON WITH IPDIFF
In this section, we compare our method with IPDiff, another diffussin model for structure-based drug design. We do not include this comparison in the main text because the training data resources are not exactly the same. Specifically, IPDiff leverages additional binding affinity data from PDBBind to pretrain a binding affinity prediction model, IPNet. This allows IPDiff to utilize protein-ligand interaction priors learned from IPNet, enabling the generation of molecules with improved target binding affinity and other properties.
We compare IPDiff with and without binding affinity pretraining, shown in Table g To fairly compare with the non- prepprerraiedd version of IPDiff, we use the same energy evaluation code as IPDiff' assess our method. This yields slightly better results than those obtained using the DiftSBDD evaluation we initially adopted. Our results show that Frag2Seq outperforms IPDiff when IPDiff is not pretrained on binding affinity data and achieves comparable Vina scores the pretrained version of IPDiff, despite Frag2Seq not utilizzig any additional binding affinity data. This further validates the effectiveness of our approach. Moreover, Frag2Seq surpasses IPDiff across all drug-likeness metrics and exhibits a generation speed approximately 64 times faster than IPDiff.
However, IPDiff still demonstrates that incorporating binding affinity prior can successfully enhance binding energy during generation. Thus, integrating binding affinity prior into the language model a promising direction for further improving Frag2Seq.
Table 8: Generation performance comparison with IPDiff. IPDiff results are taken from its paper. #We use the same energy evaluation code as IPDiff to evaluate Frag2Seq for fair comparison.
Methods
Vina Score (L)
QED (f)
SA (f)
Time (s, 4)
IPDiff w/ pretrain on binding affinity data
8.57
0.46
0.57
3063
IPDiff w/o pretrain on binding affinity data Frag2Seq
-7.55
0.52
0.61
-8.49*
0.645 - 0.15
0.642 + 0.11
48.8 1 14.6
B DERIVATION OF ROTATION VECTOR BETWEEN FRAGMENT AND MOLECULE FRAMES
In Section 3.3.2 we obtain the rotation matrix Rg-sm between fragment and molecule local frames. Then, we can derive the more compact rotation vector representation. Specifically, we can obtain the rotation angle & by
1b = cos tr (Rg-mm)
(9)
2
where tr(-) denote the trace of a matrix. Suppose Rg-tm has the following form
Mi1
T12
T13
R Rgmm-
721
722
T23
(10)
F31
F32
T33.
the rotation axis a (az, ag. ag) can be obtained by
az (ra2 /2a)/2sin v
(11)
0y (r1a r31)/2 sin vi,
(12)
ag = (r21 r v..
(13)
Then, calculate rotation vector m (Ma, my, m2) via va.
19

---

Published as a conference paper at ICLR 2025
C PROOFS
C.I PROOF OF LEMMA 3.2
Lemma (Canonical Ordering for 3D Molecular Graph Isomorphism). Let MI = Zi) and M2 = -V2, Z2) be two 3D molecular gruphs following Def. 3.7 Let M + L a function that maps molecule M E M, the set of all finite 3D molecular graphs, to its canonical order L(M) E C, the set of all possible canonical orders, as produced by the canonical SMILES. Then the following equivalence holds:
L(Mi) = (LM)) a Mi 3DD M2
where M1 SDD M2 denotes that M1 and M2 are 3D isomorphic.
Proof. We first prove the equivalence from right to left.
Given that Mi Enn M2. from Def. 3.1 there exists bijection b ver(Mi) + ver(Ma) such Mi .M, and there exists 3D transformation 7 E SE(3)
that for every atom in M1 indexed i, 2; =b()) such that n:M: -(*())) Therefore, M2 is rotated and translated instance of M1. In this case, M1 and M2 has identical molecular structure. Canonical SMILES (0 O'Boyle| 2012 Weininger et al. 1989) provides a unique string representation for a given molecular structure. We leave out the proof for the rigor of the canonical SMILES algorithm, which has been provided by O O'Boyle (2012). Thus for the canonical SMILES string of Mi and M2, $1 and $2, we have have [(M]) L(22).
82. Since L(M1) and
(M2) are the ordering in $1 and $2, respectively.
Next we prove the equivalence from left to right. For two 3D molecular graphs L(Mi) and [(M2), we know L(M)) = L(M)). According to the canonical SMILES properties, two molecules are not identical in structure, they are guaranteed to receive different canonical SMILES strings. Therefore, since L(Mi)= = L(M2), M,, and M2 must be identical in structure. Then there exists bijection b ver(Mi) + ver(M2) such that zM, = =b()
MM2 for every atom in Mi indexed i. Meanwhile, in this case, M2 can be derived from Mi after certain homogeneous 3D transformation under the right-handed Cartesian coordinate system, i.e., SE)) transformation. An arbitrary rigid transformation in SE)) can be separated into two parts, translation and rigid rotation. Let the translation and rotation needed be represented as T1, T2. SE(3) is Lie group and satisfies the axioms that the set is closed under the binary operation. T1, E SE(3). thus T172 E (3).. Let r' T1T2 (3))). Therefore, there exist a 3D transformation 7' E (3)(3) M2). Consequently, Mi an M2. such that 0z Mi (1
This completes the proof.
C.2 PROOF OF LEMMA
Lemma. Let M (V, be 3D molecular graph with node lype vector and node coordinate matrix V. Let m be equivariant local frames of M built based on the first ihree non-collinear fragment centers in L(M) and g be equivariant local frames of any fragment G, built based on the first three non-collinear atoms in L(G.). f(-) is our function that maps 3D coordinate matrix V of M to spherical representations under the molecule local frame m. g(--) is the function that maps the molecule local frame mu and fragment local frame to rotation vectors m. Then for any 3D transformation E SE(3), we have f(V) f(((V)) and g(m,g) g) = g(r g)). Given spherical representations S = f(V) and rotation vectors m g(m, g). there exist a transformation T E (3),, such that f S = TVV) and y (m)) = T (m,, g).
Proof. We first prove the (E(3) -inaarineee of the spherical representations of fragment center under molecule local frame, then prove the rotation representations between the fragment and molecule local frames are also SE(3) -nnvarian.
To simplify the proof, we assume that the molecule is centered first by moving the first atom to the origin. Let l,, fz, and fm be the indices of the first three non-collinear fragment centers in M, and
20

---

Published as a conference paper at ICLR 2025
the molecule local frame (x. y, 2) is calculated
a normalize(Urg 1ez)); normalize ((Ur... 1b. 2),
(14)
2 x: x y,
where normalize(.) denotes normalizing input vectors into unit vectors. Given any SE(3) transfor- mation т containing rotation matrix and translation vector t, we have transformed coordinate
o/ T(;)) Rv; + t. Then,
apply this transformation to the vectors in Equation [14 to
obtain the transformed basis vectors for the molecule local frame. Specifically,
x' normalize RoE, t (Rv,, +)) R(normalize(10; vt, )) Ra,
y normalize (RRo + (Rve, + +t)) x x'), normalize (R ((VEm - ve,) x)),
(15)
R R(normalize ((VEm. Vt,) x x));
Ry,
z' x' x y' = Ræ x Ry- = R(x x y) Rz.
So, the new molecule local frame is subject to the same transformation. Then, for spherical coordinates of any fragment center f(V),, that is represented as
d, l|0. ve, 12.
0r arccos ((Ve, vt,) z/di.)
(16)
tt. atan2 ((vr, 1z, y: (ve, 18, x):
we can derive the transformed spherical coordinate as
dy |Ros + t - (Rus, + 2) = ||/( - 11 )112 = ||0 -
1r,,122 -v;,
dr,;
c; arccos ( (Roe, + t-- -Ro,, + t)) 2 z'/d',) arccos (R (Ve. -vt,) Rz/de,) te,.
% atan2 ( (Rov, +t - -RRve +t)) y2, (Rve, +t- - (Ror, + +t)) x'),
(17)
alan2 [8 (Vr, - -vr,) Ry, R (Ve, - Ve,)) ) Rax),
atan2 ((Vr - Ut,) ) y, (Ve. vg,) x))
Øt:-
Thus, we show that f(V) = f(((V)) Conversely, given spherical coordinates (de,, 0c, )e.) for fragment center l,, we can obtain local Cartesian coordinates through the inverse function f-) as
[de, sfn, cos(Ør,) de, sin(Or,) sin (dr,) ), de, cos(6;,))
(18)
Then, we can reconstruct the original coordinates of fragment centers by applying the SE(() trans- formation Run -wy bm- between the molecule local frame and the world frame, which is derived in Section 3.3.2 Formally,
Vt, R + tm--w-
(19)
So, there exist a transformation E SE(3), such that
Next, we prove that the rotation vector m between the molecule local frame and the fragment local frame is also (E33)innvriiant. Suppose we have the transformation (Rm- between the molecule local frame m and the world frame w, and the transformation (Rg-sws bg between the fragment local frame g and the world frame. In Section 3.3.2] we derive the transformation (Rg-m: (m) between the fragment local frame and the molecule local frame, such that
Rg-m = Ry Rg-
1g-m = RTT (fg-vu (m--w).
(20)
Given any SE(3) transformation containing rotation matrix R and translation vector t, we have
Ry-m (RRm -w) )z (((Rgg-))))))
(21)
Rd Rd RR
(22)
- Rgg+w
R2 vu
(23)
Rgg-mm-
(24)
21

---

Published as a conference paper at ICLR 2025
So have Rg-m 7(Rg-m)) Since g(m. g) is derived from Rg-mm described in Sec- tion 3.3.3, we also have g(m. = g g)).
Conversely, given rotation vector m. m = la, where 0 is the rotation angle and a = (az, ay. az) is the unit rotation axis. Then, we can construct the kkew-symmetric matrix K from such that
convert it back to rotation matrix. Specifically, let
0
K
02
2
(25)
Oz
0
Then, we apply Rodrigues' rotation formula to obtain rotation matrix Rg- such that
Rg-mm + 0) K + (1 cos(O))
(26)
Then, we have Rg-ru = Rm wn->wwel Rg-m-- From Equation 77). we can obtain f-->m and then we can have t-- Rm-wwty-n + fm-w- So, we show that there exist SE(3) transformation (Rm-ow km- such that g (m) gm,,g)
C.3 PROOF OF THEOREM 3.4
First we establish lemma and provide its proof.
Lemma C.1. Let Mi (V1, and M2 = (V2, Z2) be two 3D molecular graphs, where 2; is the node type vector and vz is the node coordinates of the molecule M,. There exists a bijection
b ver(Mi) + ver(M2) such that for every alom in Mi indexed i, M1 ==03) Let L(M) be the canonical order of molecule M. For the fragments {9i),- of M, we construct canonical order L(M) for the k fragments, f,,".., specifically by the order of fragment's ''s first-ranked atom. Let & and E, denote the indexes of the frugment labeled i correspondingly in L(Mi) and L(M2). respectively. Let f be surjective function that maps a 3D molecule M its spherical representation and rotation vector (S, m) under the equivariant molecule local frame, and between molecule local frame and fragment local frame, respectively, following Lemma 3.3 Then the following equivalence holds:
M2
=b3))
for =1= kk. J.. g(Mi)) = = = Mi Ean M2
where Mi EDD M2 denotes that Mi and M2 are 3D isomorphic.
Proof. From Def. 3.1 we know
there exists bijection b: ver(Mi) ver(M2) s.l.Vi, z,Mi 2)() and there exists 3D transformation J E (SE(3) such that Dy. T
Mi aD M2
for i. Note that since the fragmentation process unique, deterministic, and linear, for fragments (9.)%, of M. (G.)- = M. Thus fragmentation does not affect any properties. Thus it's sufficient to prove the equivalence of the two statements on right side with
for i 1, k. o ((M)), = g(M2)
Here, the first statement on the right holds by definition. For the second condition, we establish T(x,,) = Rvs + t. Here R is a rotation matrix, and & is translation vector.
Following the proof of Lemma 3.3 in Appendix C.2 for any 3D transformation
SF(3),
have f(V) = f(T(V)) and g( g) = g g)). Thus if there exists 3D transformation 7 E
SE(3) such that M2 (b, M1), ), there exists 3D transformation E SE such that u G.M2
(( for 1.... k. and then
f ( (S1, m.) (S2, m2) = f ((M2)
for 1,... k. This proves direction of the equivalence.
22

---

Published as a conference paper at ICLR 2025
Also following the proof of Lemma 3.3 in Appendix C.2 given spherical representations S
f(V) and rotation vectors m = m. g), there exist a transformation T E SE(3), such that 1, k, if we have f g(Mi)e (S1, m1) = (S2, m2) f g(M2)e, there exist a transformation T E SB(3), such that r'(0)) for T' SE(3), and thus r'(w)) (Si) = (- = (V2). Since SE(3) is a Lie group satisfying the axioms that the set is closed under the binary operation, and that for every element in SE(3), there is an identity inverse. Let the inverse identity of T be T Then T T(2) = ='()), and thus 22 (v1) with 7" = T' E SE(3) for 1,.... This proves the other direction of the equivalence.
f (S) r(V) and 9 (m) = 7m, g). For
(S2) = =T(22) and g- (m) = T [m, 92). Similarly. (S,)
Therefore, show that M1 SDD M2 Vi g(M holds.
Then we prove Theorem 3.4
Theorem (Bijective mapping). Following Def. 3.1 let M1 = (V1, (V, Z1) and M2 = (V2, Zz) be two 3D molecular graphs. Let L(M) be the canonical order for M and f(.) and g(:;)) be the vector for a molecule fragment Ge be a. 88,, d,, Big 6b, mais Mayis m where the vector elements derived from f()) and g(:;)) as predefined. For the fragments {Gi)
functions following Lemma 3.3 Let the fragment-position
M,, we construct canonical order L(M) for the k fragments, .,.... lu. specifically by the order of a fragment's 's first-ranked atom. We define Frag2Seq: M + U,
)
which maps a molecule M M. its sequence representation U U. the set of all possible sequence representations, as
(ra = concat(a ol.)
where concat(-) concatenates elements as a sequence. Then Frag2Seq(-) a surjective function,, and the following equivalence holds:
(rag2Seq(M11) = Frag2Seq(MM)) + MM SDD M2.
where M₁ 3DD M2 denotes M1 and M2 are 3D isomorphic. we allow rounding up spherical coordinate and rotation vector values 2 b decimal places, then the surjectivity and equivalence still hold, only molecules M1 and M2 are (110~b/2) -2)cco/trane 3D) isomorphic.
we
Proof. First, we prove that Frag2Seq M U is a surjective function. Given the definition Frag2Seq concat(x)), "",,,
where xt [st, dg, Gs, .b, Mai My mzs,, we need to prove that all operations are deterministic. c- and 8; are defined to be deterministic, and following Lemma C.1 d;, 0z, dx, Mai, Mys, Mi derived from fog. which is function. L(M) outputs M's canonical order, which is deterministic from the canonical SMILES algorithm. Therefore, Frag2Seq M - 2/ is a well-defined function; given a 3D molecule, we can uniquely construct a ID sequence from Frag2Seq.
Next we prove Frag2Seq's surjectivity. Given any output sequence E l of Frag2Seq, the sequence in the format
q concat((s: d1 O1, $1,
11, m21, Myy, m.1): (................. [sk, dx. Ox: Φk. mak: Myk: mak))
For each fragment in and its vector [d;. Oi, di, mai, Myin mai), we follow Lemma 3.3 derivations. Given the surjectivity of the spherical representation function ʃ, rotation vector function g. and the defined f-1,g -1 there must be a unique M 2V, E M where
[d1, 01, Φ1, ma1, My1, mal..... [dx, Oz, db, Mzk, Mykk maa] = f g(M).
Therefore, output sequence g l4 there exists
M = (V, Z) M ie., Frag2Seq is surjective; given a sequence output of Frag2Seq, we can uniquely reconstruct a 3D molecule.
s.f. Frag2Seq(MM.
Since all the above proof does not involve numerical comparisons and 3D isomorphic, they also hold if we allow rounding up spherical coordinate and rotation vector values to b decimal places.
23

---

Published as a conference paper at ICLR 2025
Now prove the equivalence Frag2Seq(M1)) = Frag2Seq(M2)) Mi S&D M2, starting from right to left. We also cover the constrained case where we allow rounding up values to 2 decimal places. When number is truncated after b decimal places, according to the rounding principle, the
maximum error caused is v |10 If M1 EAD M2 (or M1 = 3D- طرr M2), i.e,, molecules
2
M1 and M2 are 3D isomorphic (or (10=-1/2)-consttrain
3D isomorphic), then from Lemma 3.2
we know the canonical forms [(M ((M2). For the fragments {G: of M. we construct canonical order ((M) for the fragments, é1, ... Lk.. specifically by the order of fragment's first-ranked atom. Let f; and f. denote the indexes of the fragment labeled i correspondingly in
L(Mi) and L (M2),, respectively. Since molecules Mi and M2 are 3D isomorphic (or (110- 122)- (2)- constrained 3D isomorphic). from 3.1 we know Vi of {G:)/- St, se; and from Lemma C.1
we know for = 1, ,k, f o g(Mi), f g(M2)e, (or satisfies with 110 error range allowed for each numerical value). Thus, we have
2
Frag2Seq(M1)
concat( 1. vit))
(G.) (. http::/...........a..ooooaaaadaa de, tc, . Oe,, myt, Myle Mat,)) mat,)) de; Oe, de, mzd, myfe m=r())
concat, Mtt.siAMM.....k.........oaaooooa
: :. miz))
Frag2Seq(M2).
(27)
where s.(+) is the canonical SMILES string. Therefore, we have shown that if two molecules are 3D isomorphic (or within the round-up error range _07"1), their sequences resulting from Frag2Seq must be identical.
2
Finally, we prove the equivalence from left to right. We provide proof by contradiction. Given that (M)) = Frag2Seq (M2), we assume that the molecules Mi and M2 are not 3D
isomorphic (or not |10 constraiied 3D isomorphic). We denote with M₁ = (V1, Z1) and M2 2
(V2, Z2). If M1 and M2 are not even isomorphic for Z,, then from 3.1 there does not exist node-to-node node mapping from Mi to M2, where each atom is identical in atom number. Then from Lemma 3.2] we know the canonical forms L(M1) # L(M2). Thus for Frag2Seq(M1)
concat, (G mai. Mio()..)) (ise, de HE, de, malei Mydi m=i.)) and Frag2Seq(M2)
concat, (G.)aa,, ht
de, 0e,, Se, may Mydt, mzze);
there must be at least one pair of st, sg, where St, F sp.- Therefore, Frag2Seq # Frag2Seq(M2). which is contradiction to the initial condition and ends the proof.
If M1 and M2 are isomorphic for 74, we continue with the following analyses. Let l, and E denote the indexes of the fragment labeled correspondingly in L(Mi) and L(M2), respectively. Let f fog be the surjective function mapping 3D graph to its spherical and rotation representations. Since M
and M2 are not 3D isomorphic (or not even [10-5 constrained 3D isomorphic), from Lemma C.1
know there exists at least one
E (1, s.I. g(Mi),, + S(M2)e,
|10 (or even with error range 2 allowed), otherwise, we would have
Vi (1, ,k), ((Mi),, s(M2)e; + Mi San Ma. contradicting the above condition. Thus for Frag2Seq(MM)
concat,.
(10, de, O2,, tee, mat, Myl. m2z,))
and Frag2Se((M2)
concat /tt.a//...............oeoaaaaaa. de, Oet de, mwf myff mze
24

---

Published as a conference paper at ICLR 2025
at least one pair of spherical coordinates does not correspond, there must be at least pair of (de, Be, Pt,, Mzt,, myl, mz(,) ) and (de, de, Qr, Mzx, mye, mar.) where
(de, oe. Pe Mods myle; mat,) # (dr; Prg. 06, Moll myfe matt)
(or
110-9
min(Ide,: Oz, 0E Mar,, Myge matal Ide; Oef D2g, mal, ye mae D
2
Thus, (rag2Seq(M1) Frag2Se((M2), which contradicts the initial condition. Therefore, we have shown that if two constructed sequences from Frag2Seq are identical, their corresponding molecules must be 3D isomorphic. This ends the proof.
25


Reviews:
[{"review_id": "VPfu1TSqUv", "reviewer": "Reviewer_3w4Z", "summary": "The paper presents Frag2Seq, a method for structure-based drug design using language models (LMs) to generate drug-like molecules based on 3D fragments. By transforming 3D molecules into SE(3)-invariant sequences which can used with LMs, Frag2Seq captures essential geometric information. Additionally, it incorporates protein context through cross-attention with pre-trained protein embeddings, enabling effective target-aware molecule generation. Experiments show that Frag2Seq shows competitive performance against baselines in binding affinity, drug-likeness (QED), and synthesis accessibility, with sampling speeds up to 300 times faster than atom-based generative models.", "questions": "1. How robust and adaptable are the fragmentation rules in Section 3.2 when applied to structurally complex molecules? Could this fragmentation style lead to a loss of important molecular interactions?\n2. The authors should consider providing an attention map visualization highlighting the interactions between protein pocket features and small molecule. This could  help enhance interpretability, and improve practical usability in drug discovery applications", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Frag2Seq effectively incorporates protein context through cross-attention with pre-trained protein embeddings, allowing it to generate target-specific molecules. This approach addresses a common limitation in SBDD by aligning molecular generation more closely with real-world drug discovery where small molecule and protein interactions play a vital role in binding.\n2. The SE(3)-invariant tokenization method preserves important 3D geometric information, offering a promising way to leverage the generative capabilities of LLMs. This geometric awareness could enhance accuracy in predicting drug-like properties, such as binding affinity and drug-likeness.\n3.  Frag2Seq demonstrates strong performance on metrics like binding affinity and drug-likeness while achieving up to 300x speedup over atom-based approaches.", "weaknesses": "1. For the models to be practically usable in applications like drug discovery, the scoring mechanism for generated molecules needs greater clarity on how specific molecular characteristics (e.g., fragment size, functional groups) influence metrics like vina scores. In other words the paper can focus a bit more on model interpretability.\n2. The fragment-based generation approach, which splits molecules by rotatable bonds while preserving rigid functional groups (e.g., hydroxy, carboxyl), may limit flexibility in positioning critical binding groups, potentially affecting target-specific binding. How robust are the fragmentation rules in Section 3.2?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E3LKa5Rxj7", "reviewer": "Reviewer_DBhm", "summary": "This paper proposes Frag2Seq, which leverages fragment and geometric information to enhance structure-based drug design. By transforming 3D molecules into fragment sequences and using SE(3)-equivariant frames to preserve the geometry of fragments, this model aims to improve the efficiency of molecule generation. The paper also incorporates protein pocket embeddings using cross-attention to capture protein-ligand interactions and demonstrates some performance improvements.", "questions": "1. Could the authors provide more detailed experimental results, especially in terms of binding affinity and molecular quality, to further validate the model’s effectiveness?\n2. How does this model perform on CrossDocked2020, in comparison with IPDiff?\n3. Is it possible to consider integrating an interaction prior mechanism similar to IPDiff within the diffusion model to enhance generation accuracy and practical applicability?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Innovation in fragment-based generation: The model’s use of fragment-based generation reduces the computational complexity of molecule generation.\n2. Incorporation of protein embeddings: The use of protein pocket embeddings to enhance the targeting of drug molecule generation demonstrates a promising attempt at capturing protein-ligand interactions.\n3. Generation efficiency: The method improves generation speed over traditional atom-level generation approaches.", "weaknesses": "1. Lack of innovation: The model structure closely resembles that of IPDiff, with the only significant change being the replacement of the diffusion mechanism with a language model, limiting the novelty of the approach.\n2. Insufficient binding affinity accuracy: The model underperforms in binding affinity compared to IPDiff, which achieves higher Vina scores.\n3. Lack of critical comparative analysis: There are no direct accuracy comparisons with state-of-the-art models, like IPDiff, preventing a full evaluation of the model's advantages.\n4. Limited practicality and scalability: Although fragment generation improves computational efficiency, it is limited in producing high-accuracy molecules and binding affinity, which reduces its practicality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k46Sf3mDjb", "reviewer": "Reviewer_bkQt", "summary": "The authors propose a fragment-based bijective mapping between 3D molecules and canonical SE(3)-invariant sequences. They do so by combining molecule and fragment local frames and representing fragments as spherical coordinates w.r.t to the molecule frame. They prove that the representation preserves structural completeness and geometric invariance. They adopt the representation for LM-based conditional generation of ligands for protein pockets, demonstrating competitive results with a wide set of baselines.", "questions": "- In lines 248-249, the authors write \"...with the key to be the canonical SMILES of each fragment and the value to be the atom types and atom local coordinates in each fragment under the associated fragment local frame.\" => why is the key only the canonical SMILES? Would different conformations of the same fragment be represented by the same key?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proposed sequence representation respects structural completeness and geometric invariance, and the authors provide rigorous proofs for the claims.\n- The SBDD baselines are comprehensive, with the method achieving superior results in efficacy and efficiency\n- The authors include sufficient ablations and analysis for the design choices of their method.", "weaknesses": "- The approach is not entirely novel (though its application to SBDD is)\n- It's unclear why this text-based representation and LM-based generation is needed for SBDD\n- Alternate strategies for incorporating protein context should be compared with", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FOc4j0Hah0", "reviewer": "Reviewer_AJ93", "summary": "This paper introduces Frag2Seq, a fragment and geometry-aware language model designed for structure-based drug design. The model employs a SE(3)-equivariant approach to decompose 3D molecular structures into fragments and represents these fragments as sequences while preserving their geometric information. Protein embeddings are extracted using the pre-trained inverse folding model ESM-IF1. To effectively capture protein-ligand interactions, Frag2Seq incorporates cross-attention mechanisms between the molecule embeddings and the protein embeddings within the language model. Experiments on the CrossDocked dataset show that Frag2Seq outperforms the baselines on  metrics such as Vina Score, QED, and Lipinski The method also exhibits superior sampling efficiency, with up to approximately 300 times faster generation compared to atom-based autoregressive and diffusion baselines.", "questions": "1. Since the language model does not directly generate chemical bond information, how reliable is the subsequent model used for bond inference, and how is its performance validated? Do baseline models also require bond inference, and if so, do they use the same model? Please elaborate more on the rationality behind the experimental design to ensure that the performance of drug design is accurately assessed.\n2. Could you explain why a language model was chosen over other methods, such as 3D atom-based diffusion models, for the task of structure-based drug design? What are the theoretical and practical benefits?\n3. Is the direct use of cross-attention between the entire protein and molecule sequences a simplistic approach, or can experiments demonstrate that it effectively captures key information about the binding pocket?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The method proposed in the paper significantly outperforms recent state-of-the-art methods in terms of drug-likeness and high binding affinity of the generated molecules, demonstrating its superiority in generating drug candidates.\n2. The sample efficiency of Frag2Seq is markedly higher than that of existing methods, offering substantial speedup in the generation process. This efficiency is crucial for accelerating the drug discovery pipeline and reducing computational costs.\n3. The paper innovatively leverages the established ordering with canonical SMILES for 3D-isomorphic molecules, which is of paramount importance for sequence models. This approach ensures that the fragments of molecules  are accurately represented and compared in a sequence format.", "weaknesses": "1. The use of SE(3)-invariant frames to encode the 3D structure of molecules, while effective, is a common approach in  protein structure design and thus does not introduce a significant innovation in this paper. The methodology primarily leverages the canonical ordering of SMILES strings to concatenate fragment sequences, which is a straightforward application. Similarly, the use of cross-attention to directly integrate protein and small molecule information is also quite direct. As a result, this work is more incremental in nature rather than presenting a breakthrough in methodology.\n2. In terms of input processing, the quantization of continuous structural information into discrete tokens results in a loss of precision. This could potentially affect the model's ability to capture fine-grained details of molecular structures. On the output side, the language model is unable to directly generate information about chemical bonds, which necessitates the use of additional models to infer this critical information. This dependency on secondary models for bond inference introduces an extra step in the generation process and may impact the overall coherence and accuracy of the generated molecules.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)