Equivariant Shape-Conditioned Generation of 3D Molecules for Ligand-Based Drug Design
Authors: Keir Adams
Connor W. Coley
Venue: ICLR 2023 poster (2023)
Keywords: molecules
equivariance
generation

Abstract: Shape-based virtual screening is widely used in ligand-based drug design to search chemical libraries for molecules with similar 3D shapes yet novel 2D graph structures compared to known ligands. 3D deep generative models can potentially automate this exploration of shape-conditioned 3D chemical space; however, no existing models can reliably generate geometrically realistic drug-like molecules in conformations with a specific shape. We introduce a new multimodal 3D generative model that enables shape-conditioned 3D molecular design by equivariantly encoding molecular shape and variationally encoding chemical identity. We ensure local geometric and chemical validity of generated molecules by using autoregressive fragment-based generation with heuristic bonding geometries, allowing the model to prioritize the scoring of rotatable bonds to best align the growing conformation to the target shape. We evaluate our 3D generative model in tasks relevant to drug design including shape-conditioned generation of chemically diverse molecular structures and shape-constrained molecular property optimization, demonstrating its utility over virtual screening of enumerated libraries.

Published as a conference paper at ICLR 2023
EQUIVARIANT SHAPE-CONDITIONED GENERATION OF 3D MOLECULES FOR LIGAND-BASED DRUG DESIGN
Keir Adams' & Connor W. Coley!.2 1.2
'Department of Chemical Engineering, MIT
Department of Electrical Engineering and Computer Science, MIT (keir, ccoley J Jemit edu
ABSTRACT
Shape-based virtual screening is widely used in ligand-based drug design search chemical libraries for molecules with similar 3D shapes yet novel 2DD graph struc- tures compared known ligands. 3D deep generative models can potentially automate this exploration of shape-conditioned 3D chemical space; however, no existing models can reliably generate geometrically realistic drug-like molecules in conformations with specific shape. We introduce a new multimodal 3D gen- erative model that enables conditiond 3D molecular design by equivari- antly encoding molecular shape and variationally encoding chemical identity. We ensure local geometric and chemical validity of generated molecules by using au- toregressive sigment-based generation with heuristic bonding geometries, allow- ing the model to prioritize the scoring rotatable bonds to best align the growing conformation the target shape. We evaluate our 3D generative model in tasks relevant to drug design including shape- conditcood generation of chemically di- verse molecular structures and chape-constrained molecular property optimiza- tion, demonstrating its utility over virtual screening of enumerated libraries.
INTRODUCTION
Generative models for de novo molecular generation have revolutionized computer- aided drug de- sign (CADD) by enabling efficient exploration of chemical space, goal-directed molecular optimiza- tion (MO), and automated creation of virtual chemical libraries (Segler et al.) 2018 Meyers et al.,
2021 Huang et al. 2021 Wang et al.) 2022 et al. 2022 Bilodeau et al. 2022). Recently, several 3D generative models have been proposed to directly generate low-energy or (bio)active molecular conformations using 3D convolutional networks (CNNs) (Ragoza al.] (020). reinforcement learn- ing (RL) (Simm et al. 2020a b), autoregressive generators (Gebauer et al. [2022] Luo & Jil 2022). or diffusion models (Hoogeboom et al. 2022). These methods have especially enjoyed accelerated development for structure-based drug design (SBDD), where models are trained to generate drug-
like molecules in favorable binding poses inside explicit protein pocket (Drotár al. 2021 Luo
et al. 2022 Liu et al.) 2022,] Ragoza et al. 2022). However, SBDD requires atomically-resot structures of protein target, assumes knowledge of binding sites, and often ignores dynamic pocket flexibility, rendering these methods less effective in many CADD settings.
Ligand-based drug design (LBDD) does not assume knowledge of protein structure. Instead, molecules are compared against previously identified "actives" on the basis of 3D pharmacophore or 3D shape similarity under the principle that molecules with similar structures should share similar activity (Vázquez et al., 2020 Cleves & Jain 2020). In particular, ROCS (Rapid Overlay of Chemi- cal Structures) is commonly used as shape-based virtual screening tool to identify molecules with similar shapes to reference inhibitor and has shown promising results for scaffold- caafold-hopppig tasks (Rush et al. 20055 Hawkins et al.) 2007 Nicholls et al. (2110). However, virtual screening relies on enumeration of chemical libraries, fundamentally restricting its ability to probe new chemical space.
Here, we consider the novel task of generating chemically diverse 3D molecular structures con- ditioned a molecular shape, thereby facilitating the shape-conditioned exploration of chemical space without the limitations of virtual sereening (Fig. 1). Importantly, shape- conde-condiiioned 3D molecular generation presents unique challenges not encountered in typical 2D generative models:

---

Published as a conference paper at ICLR 2023
Shape- Conditioned SD Molecular Generation
M' P(M[S)
S
Shape Similarity (1)
Chemiral Diversiyy (1)
Approach
Equvarlanty er groodo molcouarr shape
Vertationally encoce moleculer iienrity
Mix st and chemsayy codes
Generate malccules fragments heurstto bond
Score retasable bones glabal shape
NM2
Figure 1: We explore the task of shape-conditioned 3D molecular generation to generate chemically diverse molecules in 3D conformations with high shape similarity encoded target shape.
Challenge 1. 3D shape-based LBDD involves pairwise comparisons between two arbitrary confor- mations of arbitrary molecules. Whereas traditional pooerty-conditioon generative models or MO algorithms shift learned data distributions optimize single scalar property, chape-conditioned generative model must generate molecules adopting any reasonable shape encoded by the model.
Challenge 2. Shape similarity metrics that compute volume overlaps between molecules (e.g., ROCS) require the molecules to be aligned in 3D space. Unlike 2D similarity, the computed shape similarity between the molecules will change if one of the structures is rotated. This subtly impacts the learning problem: if the model encodes the target 3D shape into an 3E(3)-invariant representation, the model must learn how the generated molecule would fit the target shape under the implicit action of an SE(3) 3E(3)-aligmment. Alternatively, if the model can natively generate aligned structure, then the model can more easily learn to construct molecules that fit the target shape.
Challenge 3. A molecule's 2D graph topology and 3D shape are highly dependent; small changes in the graph can strikingly alter the shapes accessible to molecule. It thus unlikely that generative model will reliably generate chemically diverse molecules with similar shapes an encoded target without simultaneous graph and coordinate generation; and 2) explicit shape-conditioning.
Challenge 4. The distribution of shapes drug-like molecule can adopt is chiefly influenced by ro- tatable bonds, the foremost source molecular flexibility. However, existing 3D generative models mainly developed using tiny molecules (e.g., fewer than 10 heavy atoms), and cannot generate flexible drug-like molecules while maintaining chemical validity (satisfying valencies), geometric validity (non-distorted bond distances and angles; steric clashes), and chemical diversity.
To surmount these challenges, we design a new generative model, SQUID to enable the shape- conditioned generation of chemically diverse molecules in 3D. Our contributions
follows:
Given a 3D molecule with target shape, we use equivariant point cloud networks to encode the shape into (rotationally) equivariant features. We then graph neural networks (GNNs) to variationally encode chemical identity into invariant features. By mixing chemical features with equivariant shape features, we can generate diverse molecules in aligned poses that fit the shape.
We develop a sequential fragment-based based 3D generation procedure that fixes local bond lengths and angles to prioritize the scoring of rotatable bonds. By massively simplifying 3D coordinate generation, we generate drug-like molecules while maintaining chemical and geometric validity.
We design a rotatable bond scoring network that learns how local bond rotations affect global shape, enabling our decoder to generate 3D conformations that best fit the target shape.
We evaluate the utility of SQUID over virtual screening in shape-conditioned 3D molecular design tasks that mimic ligand-based drug design objectives, including shape-conditioned generation of diverse 3D structures and shape-constrained molecular optimization. To inspire further research, we note that our tasks could also be approached with a hypothetical 3D generative model that dis- entangles latent variables controlling 2D chemical identity and 3D shape, thus enabling zero-shot generation of topologically distinct molecules with similar shapes to any encoded target.
'SQUID: Shade-Conditioned Equivariant Generator for Drug-Like Molecules
2

---

Published as a conference paper at ICLR 2023
2 RELATED WORK
Fragment-based molecular generation. Seminal works in autoregressive molecular generation ap- plied language models to generate ID SMILES strings character- saaractee-by-chareee by-character (Gomez- Bombarell
al., 2018; Segler et al. 2018), or GNNs to generate 2D molecular graphs atom-by-aton (Liu et al. 2018; Siimnnvssy & Komodakis, 2018 Li et al.) (0018). Recent works construct molecules fragment-by-fragmeen to improve the chemical validity of intermediate graphs and to scale genera- tion to larger molecules (Podda et al. 20200 Jin et al., 2019 2020p. Our fragment-basee decoder is related to MoLeR (Maziarz et al. 2022). which iteratively generates molecules by selecting a new fragment (or atom) to add to the partial graph, choosing attachment sites on the new fragment, and predicting new bonds to the partial graph. Yet, MoLeR only generates 2D graphs; we generate 3D molecular structures. Beyond 2D generation. (Flmm Sham-hh et al. (2022) an RL agent gen- erate 3D molecules by sampling and connecting molecular fragments. However, they sample from small multiset of fragments, restricting the accessible chemical space. [Powers et al. (2022) use fragments to generate 3D molecules inside a protein pocket, but only consider 7 distinct rings.
Generation of drug-like molecules in 3D. In this work, we generate novel drug-like 3D molecular structures in free space, e.g., not conformers given a known molecular graph (Ganea et al. 2021 Jing et al. 2022). Myriad models have been proposed generate small 3D molecules such as E(3)-equivariant normalizing flows and diffusion models (Satorras et al., 2022a Hoogeboom et al., 2022]. RL agents with an SE(3)-covariant action space (Simm et al., 2020b), and autoregressive generators that build molecules atom-by-atom with SE(3)-invariant internal coordinates (Luo & Ji. 2022 Gebauer et al., 2022). However, fewer 3D generative models can generate larger drug-like molecules for realistic chemical design tasks. Of these, Hoogeboom et al. (2022) and Arcidiacono & Koes (02I) fail to generate chemically valid molecules, while Ragoza et al. (2020) rely on post- processing and geometry relaxation to extract stable molecules from their generated atom density grids. Only Roney et al. (2021) and Li et al. (022). who develop autoregressive generators that si- multaneously predict graph structure and internal coordinates, have shown to reliably generate valid drug-like molecules. We also couple graph generation with 3D coordinate prediction; however, we employ Traseettbbased generation with fixed local geometries to ensure local chemical and geo- metric validity. Futher, we focus on shape-conditioned molecular design; none of these works can natively address the aforementioned challenges posed by shape-conditions molecular generation.
Shape-conditioned molecular generation. Other works partially address shape-conditioned 3D molecular generation. [Skalic et al. (2019) and [imrie et al. (202T) train networks to generate 1D SMILES strings or 2D molecular graphs conditioned on CNN encodings of 3D pharmacophores. However, they do not generate 3D structures, and the CNNs do not respect Euclidean symmetries. Zheng et al. (202T) use supervised molecule-to-mokee mollecule translation on SMILES strings for scaf- fold hopping tasks, but do not generate 3D structures. PPapadopoulos et al. (202T) use REINVENT Olivecrona et al. 2017) SMILES strings to propose molecules whose conformers are shape- similar to a target, but they must re-optimize the agent for each target shape. Roney et al.] (2021) fine-tune 3D generative model on the hits of ROCS virtual screen of > 10.0 drug-like molecules to shift the learned distribution towards a target shape. Yet, this expensive screening approach must be repeated for each new target. Instead, we seek to achieve zero-shot generation of 3D molecules with similar shapes to any encoded shape, without requiring fine-tuning or post facto optimization.
Equivariant geometric deep learning on point clouds. Various equivariant networks have been designed to encode point clouds for updating coordinates in Rdd (Satorras et al. 2022b), predicting tensorial properties (Thomas et al., 22118). or modeling 3D structures natively Cartesian space (Fuchs et al. 2020). Especially noteworthy are architectures which lift scalar neuron features to vector features in Th and employ simple operations to mix invariant and equivariant features with- out relying on expensive higher-order tensor products or flebsh-Gordan coefficients (Deng et al.) 2021 [Jing et al. 202I). In this work, we employ [eng et al.) (202l)ss Vector Neurons (bNed equivariant point cloud encoder VN-DGCNN to encode molecules into equivariant latent represen- tations in order to generate molecules which are natively aligned to the target shape. Two recent works also employ VN operations for structure-bast based drug design and linker design (Peng et al.) 2022 Huang et al. 2022). Huang et al. (2022) also build molecules in free space; however, they generate just few atoms to connect existing fragments and do not condition molecular shape.
3

---

Published as a conference paper at ICLR 2023
3 METHODOLOGY
Problem definition. We model a conditional distribution P(M[S) over 3D molecules M M (G, G) with graph G and atomic coordinates G = (rre E given
given a 3D molecular shape S. Specifically,
we aim to sample molecules M' ~ P(M[S) with high shape similarity (sims(M', .Ms) ~ 1) and low graph (chemical) similarity (sima (M' .Ms) < 1) to a target molecule Mg with shape S. This scheme differs from 1) typical 3D generative models that learn P(M) without modeling P(M[S), and from 2) conditiond ID/2D generators that attempt to model P(G]S), the distribution of molccular graphs that could adopt shape S. but do not actually generate specific 3D conformations.
We define graph (chemical) similarity sime E [0, 1J between two molecules as the Tanimoto sim- ilarity computed by RDKit with default settings (2048-bit bit fingerprints). We define shape similar- ity sims E [0, 1] using Gaussian descriptions of molecular shape, modeling atoms E M and b E MB from molecules MA and MB as isotropic Gaussians R& (Grant & Pickup/ 1995 Grant et al. 1996). We compute simy using (2-body) volume overlaps between atom-centered Gaussians:
VAB
Σ Vasi Vas x exp -- -oolllll)) (I)
(G GB)
VAB =
VAA + BBB VAB
BEA.BEB
where a controls the Gaussian width. Setting a = 0.81 approximates the shape similarity function
used by the ROCS program (App. (A.)). sim' is sensitive to SE(3) transformations of molecule MA with respect to molecule MB. Thus, we define sims(M MB) = max (G GB) as the R.t
shape similarity when MA is optimally aligned to MB. We perform such alignments with ROCS.
Approach. At a high level, we model P(M[S) with an encoder-decoder architecture. Given a molecule Ms = (Gs, Gs) with shape S, we encode S (a point cloud) into equivariant features. We then variationally encode Gs into atomic features, conditioned on the shape features. We then mix these shape and atom features to pass global SE(3) {in.equi} variant latent codes to the de- coder, which samples new molecules from P(M)S). We autoregressively generate molecules by factoring P(M|S) PM P(M Mo (.)..P(M|MMM..1. S), where each Mj (Gi, G1) are partial molecules defined by a BFS traversal of a tree-representatiion of the molecular graph (Fig. 2). Tree-nodes denote either non-ring atoms or rigid (ring-containing) fragments, and tree-links denote acyclic (rotatable, double, or triple) bonds. We generate Mi+1 by growing the graph G around a focus atom/fragment, and then predict G1+1 by scoring query rotatable bond best fit shape S.
variiat
Simplifying assumptions. (1) We ignore hydrogens and only consider heavy atoms, as is common in molecular generation. (2) We only consider molecules with fragments present in our fragment library to ensure that graph generation can be expressed as tree generation. (3) Rather than gener- ating all coordinates, we use rigid fragments, fix bond distances, and set bond angles according to hybridization heuristics (App. A.8); this lets the model focus scoring rotatable bonds to best fit the growing conformer to the encoded shape. (4) We seed generation with Mo (the root tree-node), restricted to be small (3-6 atoms) substructure from Ms; hence, only model SS Mo).
3.1 ENCODER
Featurization. We construct a molecular graph using atoms as nodes and bonds as edges. We featurize each node with the atomic mass; one-hot codes of atomic number, charge, and aromaticity; and one-hot codes the number of single, double, aromatic, and triple bonds the atom forms (in- cluding bonds to implicit hydrogens). This helps us fix bond angles during generation (App. A.8) We featurize each edge with one-hot codes of bond order. We represent a shape S as a point cloud built by sampling nu points from each of nA ctom-centered Gaussians with (adjustable) variance op.
Fragment encoder. We also featurize each node with a learned embedding E Rd of the atom//ragmmen type to which that atom belongs, making each node "fragment-awar MoLeR). principle, fragments could be any rigid substructure with 2 2 atoms. Here, we specify fragments as ring-containing substructures without acyclic single bonds (Fig. 14.. We construct library Ly of atom/fragment types by extracting the top-k (k- = 100) most frequent fragments from the dataset and adding these, along with each distinct atom type, to y (App. 1313). We then encode each atom/fragment in L, with a simple GNN (App. A.12) to yield the global atom/fragment em-
(similar to
beddings: (f: = Ea h (a) (h GNN c, (G),) Af. C where h (()) are per-atom features.
4

---

Published as a conference paper at ICLR 2023
VN-DGONN
Wir
X
Z
Encoder
Ida. 3
Point Cloud
X
Wii II
WLP Rechape
ost
HLP
H (ou
Z
(d.)
Hu (m
H.
GNN
MLP
Hyar
(Sta,
H (Bi.
N(0.1)
(n..ch.)
Sent to Decoder
Molecular Graph
VN-DGICNN
XH
Σ
Z'
7
Decoder
MWM
Tocus
NLP + Reshupe
Partial Graph Point Cloud
Wii
NLP
X H
(d.)
Generation Sequence
Generation Tree
Z
7
HiM
14..
Zdec
x
focall
unacored
MUN
Recus band
Encode
Zelc
Stop
Cirow
Grow
Sedect tocal attachement
Fggh
Select neet atom/tramment
Choocs attachment
Predict bond order
SINGLE
not encoded
until in tocus
scoring
Stop of
Score quarry band rotation
Pap next focus
Hih?
Altach
Fidlect
Grow
Stop
Add to queue
Figure 2: Encoder-decoder architecture of SQUID, which equivariantly encodes molecular shape and variationally encodes chemical identity to generate chemically diverse 3D molecules that fit the shape. SQUID generates molecules atom-by-atom and fragment-by saatt---ffrrgeentt iteratively growing the molecular graph in a tree-expansion and generating 3D coordinates by scoring rotatable bonds.
Shape encoder.
Given Ms with ny heavy atoms, we use VN-DGCNN (App- A.11] to encode the molecular point cloud Ps (x3 into set of equivariant per-point vector features Xp. xxxx3 We then locally mean-pool the np equivariant features per atom:
Xp VN-DGCNN(Ps): X x= L=)c))Pool((xg),
(2)
where X E X are per-atom equivariant representations of the molecular shape. Be- cause VN operations SO(3)eqquivariant, rotating the point cloud will rotate X: XR Although VN operations are strictly SO(3)-equivariant, we sub- tract the molecule's 's centroid from the atomic coordinates prior to encoding, making X effectively SE(3)-equivariant. Throughout this work, we denote SO(3)-equivariant vector features with tildes.
Variational graph encoder. To model P(M[S), we first use GNN (App. A.12 to encode Gs into learedd atom embeddings H (h(") (a) V a E Gs}. We condition the GNN on per-atom invariant shape features X {x(a)) E Th x6, which we form by passing X through VN-Inv (App. A.11
H GNN((H, X); Gs); X = VN-Inv(X),
(3)
where Ho E TR tdy) are the set of initial atom features concatenated with the learned fragment embeddings, H E Rua xds and (:, denotes concatenation in the feature dimension. For each atom
Ms, we then encode ln h 22 = (L)(h(a)) and sample (()) N N(h ..h h(")):
(a) = h(a)) + c(a) (a). h(a) = exp(, h (o A aEGS}.
H
(4)
where e(a) ~ (O, 1) E Rdd, Hvar E R Rd,,xd, and @ denotes elementwise multiplication. Here, the second argument of N(.,)) is the standard deviation vector of the diagonal covariance matrix.
Mixing shape and variational features. The variational atom features H are insensitive to rota- tions of S. However, we desire the decoder to construct molecules in poses that are natively aligned to S (Challenge 2). We achieve this by conditioning the decoder on an equivariant latent represen- tation of PSM)S) that mixes both shape and chemical information. Specifically, mix H. with
5

---

Given Mg, sample new molecules M' P, Mo) by encoding Ps into equivariant shape features X, variationally sampling ( for each atom Ms, mixing H with X, and passing the resultant (Z, z) to the decoder. We seed generation with a small structure My (extracted from Ms), and build M' by sequentially generating larger structures M/41 in a tree-like manner (Fig- 2). Specifically, popped from a BFS queue. To generate M/+1 from Mj (e.g., grow the tree from the focus), we factor P(M++1|AM S) = P(G++1|M, () Mr, S). Given (Z, z), sample the new graph G/+1 by iteratively attaching (a variable) C new atoms/fragments (children tree-nodes) around the focus, yielding Gy(e) for c ..., C, where G(CC) G/++1 and G Gi-- We then generate coordinates 111 by scoring the (rotatable) bond between the focus and its parent tree-node. New Published as a conference paper at ICLR 2023
X by encoding each had Haar into linear transformations, which applied atom-wise X. We then pass the mixed equívariant features through a separate VN-MLP (App. A.11I:
H.. VN-MLP(WH ( (()) x(
W H
Reshape(MLP(hh (()))) A VaeGs]. (5)
where W(a) E R (()) Rix3 and XH.. E Ruaxd, This maintains equivariance since w are rotationally invariant and W R) (w w sum-pooo pool the per-atom features in XH.,, into a global equivariant representation Z R We also embed global invariant representation the output with Hars passing through an MLP, and sum-pooling the resultant per-atom features:
H )R for rotation R. Finally, we
R by applying VN-Inv XH... concatenating
Z
MLP (a) H.. b());;
(a) XH
VN-Inv (a) H.
(6)
Hos
3.2 DECODER
grow new atoms/fragmentt around "focus" atom/fragment in Mí, which is
bonds from the focus to its children are left unscored in Mi_1 until the children become "in focus".
Partial molecule encoder. Before bonding each new atom/fragment to the focus (or scoring bonds), we encode the partial molecule M/(e-3 with the same scheme as for Ms (using parallel encoder; Fig. 2), except we do not variationally embed H' Instead, we process H' analogously to H Further, in addition globally pooling the per-atom embeddings obtain Z' E. (()) and
z' = Eo Ma) we also selectively sum-pool the embeddings of the atom(s) in focus, yielding B/one = Eaefouu X(aa and e//x =a xH
ria). We then align the equivariant representations of
M, (c-1) and Ms by concatenating Ż. Z', Z Z', and Ziee and passing these through a VN-MLP:
Zas: = VN-MLP ž', Z Z
Note that Zat E R2x3 is equivariant to rotations of the overall system (M((-)), .Ms). Mss) Finally, we form global invariant feature Zuec da condition graph (or coordinate) generation:
Zdc = (VN-Inv (Zice) z, z', z', zlpe)
(8)
Graph generation. We factor P(G [M, S) into sequence of generation steps by which we iteratively connect children atoms/fragments to the focus until the network generates (local) stop token. Fig. 2 sketches a generation sequence by which new atom/fragment is attached to the focus, yielding Gf(() from Gh( (((c-1) Given Ziec. the model first predicts whether to stop (local) generation via Po = sigmoid(MLP)) (2) E (0, 1). Po 2 To (a threshold, App. A. we stop and proceed to bond scoring. Otherwise, select which atom afoc the focus (if multiple) grow from:
((a) V focus)). Pircus softmax Peaftmax(MMLP) (MLP Focus (Zane XH
The decoder then predicts which atom/fragment Snext f to connect to the focus next:
Pucxt = softmax (MLP ( xH (am) f.
(10)
We have dropped the (c) notation for clarity. However, each Zuee is specific each (M/("-1) Ms) system.

---

Published as a conference paper at ICLR 2023
If the selected fuea is fragment, we predict the attachment site Oaite the fragment G fusi
Psite softmax ([ML.Pstte site (Zidec XH fnexts h (a) a E freu)): froe
Manc)
(11)
where h (a)
Free are the encoded atom features for Gura Lastly, we predict the bond order (1², 29, 39)
[(ane) via Phond = softmax(MLP hond (Zdec: XH fnext, h fass )). We repeat this sequence of steps until Pp 2 TØ. yielding Gi+1- At each step, we greedily select the action after masking actions that violate known chemical valence rules. After each sequence, we bond new atom or fragment to the focus, giving G((s) If an atom, the atom's position relative to the focus is fixed by heuristic bonding geometries (App. A.8). If a fragment, the position of the attachment site is fixed, but the dihedral of the new bond is yet unknown. Thus, in subsequent generation steps we only encode the attachment site and mask the remaining atoms in the new fragment until that fragment is "in focus" (Fig. 2)- This means that prior to bond scoring, the rotation angle of the focus is random. To account for this when training (with teacher forcing), we randomize the focal dihedral when encoding each M, (M1())
Scoring rotatable bonds. After sampling G/+11 P(G S), we generate G/ by scoring the rotation angle 4/411 of the bond connecting the focus to its parent node in the generation tree (Fig. 2). Since we ultimately seek to maximize sims(M',
we exploit the fact that our model simg (G (G"(G))) Gs) for various query di- 2/12.1443"*
generates shape-aligned structures to predict max
hedrals 2//41 eo of the focus rotatable bond in supervised regression setting. Intuitively, the scorer is trained to predict how the choice of the affects the maximum possible shape similarity of the final molecule M' to the target Mg under optimal policy. App. A.2 details how regression targets are computed. During generation. we sweep over each query toYoc |- r), encode each (1%)
((hae) into dec, scoter
resultant structure M41
and select the t/m that maximizes the predicted score:
(1hte) 1/4 mag max sigmoid (MLP dede, scorer)
(12)
Vite
At generation time, we also score chirality by enumerating stereoisomers G E Gion of the focus and selecting the (Gr t'toe) that maximizes Eq. 12 (App. A.2
Training, We supervise each step of graph generation with a multi-component loss function:
Lgraph-gen = Lg + Liocus + Lmeat + Late + Loond + + Phextshape meext-hhape + G shape La shape (13)
Lg, Liauss Lngeut and Lpand are standard cross-entropy losses. Lake log Ea P Psse I > 0)) is modified cross-entropy loss that accounts for symmetric attachment sites in the fragments G₁, E Ly,
(a)
where plu) are the predicted attachment-site probabilities and c, are multi-hot class probabilities. LK is the KL-divergence between the learned N (hy,. h,) and the prior (O0,,)) We also employ two auxiliary losses Lteettstupe and Ly-shupe in order to 1) help the generator distinguish between incorrect shape-similar (near-miss) vs. shape- dissimilai fragments, and 2) encourage the generator to generate structures that fill the entire target shape (App. A0 We train the rotatable bond scorer separately from the generator with an MSE regression loss. See App. A.15 for training details.
4 EXPERIMENTS
Dataset. We train SQUID with drug-like molecules (up to n4 27) from MOSES (Polykovskiy et al. 2020) using their train/test sets. L, includes 100 fragments extracted from the dataset and 24 atom types. We remove molecules that contain excluded fragments. For remaining molecules, we generate a 3D conformer with RDKit, set acyclic bond distances to their empirical means, and fix acyelic bond angles using heuristic rules. While this 3D manipulation neglects distorted bonding geometries in real molecules, the global shapes are marginally impacted, and we may recover refined geometries without seriously altering the shape (App. A. The final dataset contains 1.3M 3D molecules, partitioned into 80/20 train/validation splits. The test set contains 147K 3D molecules.
We train the scorer independently from the graph generator, but with a parallel architecture. Hence, Ziec # dec, scorer. The main architectural difference between the two models (graph generator and scorer) is that we do variationally encode Hacune into H some, as find it does impact empirical performance.
7

---

Published as a conference paper at ICLR 2023
Denast
0.3
RA6-00 -1
Mam-2
Moo Sampling Scherm
Map -2
Man -0
Chemical Siilarty
1222 OITY Kiil
1.0
62 Charical Similrrty
671)
28 O911
CON.
JIZE 2
(921.3m)
Figure 3: (A) Distributions of Sims(M, Ms) for the best of Nmax molecules with simg Ms) < 0.7 or 0.3 when sampling from the dataset (grey) or SQUID with either À = 1.0 (prior; red) or A = 0.3 (green). (B) Histograms of sims vs. simg for two sampling schemes from (A). (C) Three sets of samples from the prior (red) and the dataset (Nmsx 20; grey), overlaid on Ms (blue).
In the following experiments, we only consider molecules Ms for which we can extract small (3-6 atoms) 3D substructure Mo containing terminal atom, which principle, Mo could include larger structures from Ms, c.g., for seaffold-constraing tasks. Here, we the smallest substructures ensure that the crape-conditioned generation tasks not trivial.
to seed generation. In
Shape-conditioned generation of chemically diverse molecules. "Scaffold "Scaffold-hopping" hoppinn"" molecules with high 3D shape similarity but novel 2D graph topology compared to known inhibitors-is pursued in LBDD to develop chemical lead series, optimize drug activity, or evade in- tellectual property restrictions (Hu et al. 2017). We imitate this task by evaluating SQUID's ability to generate molecules M' with high sim M", , Mss) but low simg M Ms). Specifically, for 1000 molccules Ms with target shapes S in the test set, we use SQUID generate 50 molecules per Ms. To generate chemically diverse species, we linearly interpolate between the posterior NN (h,,. h.) and the prior (O,, 1), sampling each h N((1 - - (1 A)h, + A1) using either X 0.3 or X = 1.0 (prior). We then filter the generated molecules to have simg (M', Ms) < 0.7, or < 0.3 to only evaluate molecules with substantial chemical differences compared to Ms. Of the filtered molecules, we randomly choose Nmux samples and select the sample with highest sims Mss.
Figure 3A plots distributions of sims (M', Ms) between the selected molecules and their respective target shapes, using different sampling (Nvaa 1, 20) and filtering (simg(M'' Ms) < 0.7, schemes. We compare against analogously sampling random 3D molecules from the training set. Overall, SQUID generates diverse 3D molecules that are quantitatively enriched in shape similar- ity compared to molecules sampled from the dataset, particularly for Nmsx = 20. Qualitatively, the molecules generated by SQUID have significantly more atoms which directly overlap with the atoms of Ms. even in cases where the computed shape similarity is comparable between SQUID- generated molecules and molecules sampled from the dataset (Fig. 3C). We quantitatively explore this observation in App. A.7 We also find that using À = 0.3 yields greater sim (M', MM) than 1.0, part because using À 0.3 yields less chemically diverse molecules (Fig. 3B; Chal- lenge 3). Even sampling N = 20 molecules from the prior with (M' Ms) ) < 0.3 still yields more shape-similar molecules than sampling Nmax 500 molecules from the dataset. We emphasize that 99% of samples from the prior are novel, 95% are unique, and 100% are chemically valid (App. A.4). Moreover, 87% of generated structures do not have any steric clashes (App. A.4). indicating that SQUID generates realistic 3D geometries of the flexible drug-like molecules.
Ablating equivariance. SQUID's success in 3D shape-conditioned molecular generation is partly attributable to SQUID aligning the generated structures to the target shape in equivariant feature space (Eq. 7) which enables SQUID to generate 3D structures that fit the target shape without having implicitly learn how align structures in R3 (Challenge 2). We explicitly validate this design choice setting Ž 0 in Eq. 7 which prevents the decoder from accessing the 3D orientation of Ms during traininggeenrattion. As expected, ablating SQUID's ' equivariance reduces the enrichment in shape similarity (relative to the dataset baseline) by as much 33% (App. A.9
8

---

Published as a conference paper at ICLR 2023
Table 1: Top-1 optimized objective scores across 6 objectives and 8 seed molecules My (per ob- jective), under the shape-similarity constraint simg(M", (M) 2 0.85. (-) indicates that SQUID, limited 31K generated samples, virtual screening (VS) could not improve upon Ms.
Top-Scoring Seed Molecules
Top-Scoring (large)
Random (large)
Seed Molecules
Seed Molecules
Objective (t)
Method
A
C
D
F
G
Seed
0.69
0.69
0.61
0.48
0.50
0.42
0.08
0.02
GSK3B
SQUID
0.70
0.52
0.68
0.66
0.46
0.53
VS
0.90
0.59
0.10
0.39
Seed 0.86
0.46
0.46
0.27
0.31
0.27
0.00
0.00
JNK3
SQUID
0.95
0.63
0.63
0.36
0.32
0.23
0.25
VS
0.91
0.03
0.21
Seed SQUID
0.81
0.80
0.79
0.80
0.76
0.75
0.08
0.09
Osimertinib MPO
0.83
0.82
0.83
0.80
0.80
0.77
0.78
VS
0.79
0.46
0.78
Seed SQUID
0.45
0.44
0.44
0.25
0.23
0.24
0.00
0.00
Sitagliptin MPO
0.45
0.32
0.33
0.39
0.30
0.22
VS
0.28
0.33
0.23
0.37
Seed SQUID
0.43
0.42
0.41
0.33
0.34
0.33
0.15
0.21
Celecoxib Rediscovery
0.43
0.42
0.35
0.38
0.36
0.26
0.24
VS
0.45
0.40
0.42
0.26
0.37
Seed SQUID
0.42
0.39
0.37
0.29
0.27
0.30
0.19
0.19
Thiothixene
0.52
0.46
0.42
0.31
0.32
0.26
0.27
Rediscovery
VS
0.40
0.42
0.30
0.32
0.22
0.32
Shape-constrained molecular optimization. Scaffold-hopping is often goal-directed; e.g., aim- ing to reduce toxicity or improve bioactivity hit compound without altering its 3D shape. We mimic this shape-constrained MO setting by applying SQUID optimize objectives from Gau- caMol (Brown et al. 2019) while preserving high shape similarity (sim Ms) 2 0.85) to vari- ous "hit" 3D molecules Ms from the test set. This task considerably differs from typical MO tasks, which optimize objectives without constraining 3D shape and without generating 3D structures.
To adapt SQUID shape-constrained MO, we implement genetic algorithm (App. A.5) that iteratively mutates the variational atom embeddings H of encoded seed molecules ("hits" "i Ms in order to generate 3D molecules M* with improved objective scores, but which still the shape of Mş. Table reports the optimized top-1 scores across 6 objectives and 8 seed molecules Ms (per objective, sampled from the test set), constrained such that simg(M" Ms) > 0.85. We compare against the score of Ms. well as the (shape-constrained) top-1 score obtained by virtual screening (VS) our training dataset (> 1M 3D molecules). Of the 8 seeds Ms per objective, 3 were selected from top-scoring molecules to serve hypothetical "hits", were selected from top-scoring large molecules (2 26 heavy atoms), and 2 were randomly selected from all large molecules.
In 40/48 tasks, SQUID improves the objective score of the seed Mg while maintaining (MM" Mg) 2 0.85. Qualitatively, SQUID optimizes the objectives through chemical alter- ations such as adding/deleting individual atoms, switching bonding patterns, or replacing entire substructures all while generating 3D structures that fit the target shape (App. A.5 In 29/40 of successful cases, SQUID (limited 31K samples) surpasses the baseline of virtual screening 1M molecules. demonstrating the ability to efficiently explore new shape-constrained chemical space.
5 CONCLUSION
We designed novel 3D generative model, SQUID, to enable shape-conditioned exploration of chemically diverse molecular space. SQUID generates realistic geometries of larger molecules that are chemically valid, and uniquely exploits equivariant operations to construct conformations that fit a target 3D shape. We envision our model, alongside future work, will advance creative shape-based drug design tasks such 3D scaffold hopping and shape-constrained 3D ligand design.

---

Published as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
We have taken care facilitate the reproduciblility of this work by detailing the precise architec- ture of SQUID throughout the main text; we also provide extensive details on training protocols, model parameters, and further evaluations in the Appendices. Our source code can be found at https: / com/ kei radams/ /SOua SQUID Beyond the model implementation, our code includes links to access our datasets, as well as scripts to process the training dataset, train the model, and evaluate our trained models across the shape-conditioned generation and shape-constrained op- timization tasks described in this paper.
ETHICS STATEMENT
Advancing the shape-conditioned 3D generative modeling of drug-like molecules has the potential to accelerate pharmaceutical drug design, showing particular promise for drug discovery campaigns involving scaffold hopping, hit expansion, the discovery of novel ligand analogues. However, such advancements could also be exploited for nefarious pharmaceutical research and harmful bio- logical applications.
ACKNOWLEDGMENTS
This research supported by the Office of Naval Research under grant number N00014-21-1 2195. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. 2141064. The authors acknowledge the MIT SuperCloud and Lincoln Laboratory Supercomputing Center for providing HPC resources that have contributed to the research results reported within this poster. The authors thank Rocío Mercado, Sam Goldman, Wenhao Gao, and Lagnajit Pattanaik for providing helpful suggestions regarding the content and presentation of this paper.
REFERENCES
ROCS, Openeye Scientific Software, Cadence Molecular Sciences.
URL http: //wp:
eyesopen. com/ rocs
Michael Arcidiacono and David Ryan Koes. Molecules in 3D Space. November 2021. URL
MOLUCINATE:
Generative Model for
wwtp/bs/210....../// 15308
MXXiv:2109.15308 [q-bio].
Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F. Jensen. Genera- tive models for molecular discovery: Recent advances and challenges. WIREs Computational Molecular Science, 12(5)x1108, 2022. URL https:://onlieelibraayy wiley...w....................e.. com/
doi a03//bb//Da 10. 1002 10000.110//wee.. wcms 1608
Nathan Brown, Marco Fiscato, Marwin H.S. Segler, and Alain C. Vaucher. GuacaMol: Benchmark- ing Models for Novo Molecular Design. Journal of Chemical Information and Modeling, 59 (3):1096-118) 108, March 2019. URL http://toooo/....////oo//o 1021 /acl.a/cc... jcim. 8b00839
Ann E. Cleves and Ajay N. Jain. Structure- and Ligand-Based Virtual Screening on DUD-E+: Performance Dependence on Approximations to the Binding Pocket. Journal of Chemical In- formation and Modeling, 60(9):4296 -31129666111,,, September 2020.
pitttp::/doi..or
1021/acs jcim. 0c00115
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas Guibas. Vector Neurons: A General Framework for S0(3)-Equivariam Networks, April 2021.
URL http:://pa/xii... wwwpexivv..ggaaaoo abs/21b4 12229 arXiv:2104.1221 12229 [cs].
Pavol Drotár, Arian Rokkum Jamash, Ben Day, Cätälina Cangea, and Pietro Liò. Structure-aware generation of drug-like molecules. November 2021. Ut..ittapp///axii.oorr/bbbb// 04107 1rXiv:2111.00 11.04107 [cs. q-bio].
Yuanqi Du, Tianfan Fu, Jimeng Sun, and Shengchao Liu. MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design, March 2022. URL http: WRRLaatt:///axiiii http///xxxvvv///// abs/ 22wp.2203..2 14500
arXiv:2203. 14500 [cs, q-bio].
10

---

Published as a conference paper at ICLR 2023
Daniel Flam- Shenhhhppeer, Alexander Zhigalin, and Alán Aspuru- Guzik. Scalable Fragment- Based 3D Molecular Design with Reinforcement Learning, February 2022. URL ptttp:///axi....rrrrre
abs/ /oss/200..o 00658 arXiv:2202.00658 [es].
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. SE(3)-transformers: 3D roto- translation equivariant attention networks. 34th Conference on Neural Information Processing Systems (NeurIPS), 2020.
Octavian-Fugen Ganea, Lagnajit Pattanaik, Connor W. Coley, Regina Barzilay, Klavs Jensen, William H. Green, and Tommi S. Jaakkola. GeoMol: Torsional Geometric Generation of Molec- ular 3D Conformer Ensembles, June 2021. URL http://awww.....g/////00...... MrXiv:2106.07800 [physics].
Niklas W. A. Gebauer, Michael Gastegger, Stefaan S. P. Hessmann, Klaus-Robert Müller, and Kristof Schütt. Inverse design of 3d molecular structures with conditional generative neural net- works. Nature Communications, 1377), December 2022. URL https::t//// nature.
com/ artticeee//4146100222 022-2226-22269
J A Grant and B T Pickup. A Gaussian Description of Molecular Shape. The Journal of Chemical Physics, 99:3503-3510, 1995.
J. A. Grant, M. A. Gallardo, and B. T. Pickup. A fast method of molecular shape comparison:
A simple application of Gaussian description of molecular shape. Journal of Computational Chemistry, 17(14))6633-166, 1996. URL hettos://onlinelbrraaze
wwwey. com/ 3A 001///////O11000//22111111322223 htt.://SS....1111100000001111113
Rafael Gómezz-Bobarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-LLobtoo, Benjamíí Säncheez-Lngellnng, Dennis Sheberla, Jorge Aguiaera--parrguuiree Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik. Automatic Chemical Design Using a Data-Driven Con- tinuous Representation of Molecules. ACS Central Science, 4(2):268 276, February 2018. URL
https://doi. org/1. 11t1//22c////eeeeee 7b00572
Paul C. D. Hawkins, A. Geoffrey Skillman, and Anthony Nicholls. Comparison of Shape-Matching and Docking Virtual Screening Tools. Journal of Medicinal Chemistry, 50(1):74-82, January 2007. URL
acs ort/://///o//oooooooooo/oo/oo 10. 10210 10.1102/mm00003366 jm0603365
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant Dif- fusion for Molecule Generation in 3D, June 2022. URL 17003 arXiv:2203.170w [cs, q-bio, stat].
Ye Hu, Dagmar Stumpfe, and Jürgen Bajorath. Recent Advances in Scaffold Hopping. Journal of Medicinal Chemistry, 60(4): 1234):1238-42 1246, February 2017. URL. https://doi http///oi.rrg//.../ or0 1021/ace jmedchem. 6000m.6004444 6b01437
Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovee, Connor W. Co- ley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Therapeutics Data Commons: Machine learning datasets and tasks for drug discovery and development. In Proceedings of Neural Information Processing Systems, NeurIPS Datasets and Benchmarks, 2021.
Yinan Huang, Xingang Peng, Jianzhu Ma, and Muhan Zhang. 3DLinker: An B(3) Equivariant Variational Autoencoder for Molecular Linker Design, May 2022. / /ttp///axvvv.o///////////// abs/ 2ass/2205..m 07309 arXiv:2205.0733 [cs, bioi..
Fergus Imrie, Thomas Hadfield, Anthony R. Bradley, and Charlotte M. Deane. Deep generative design with 3D pharmacophoric constraints. Chemical Science, 12(43): 2021. URL
http: http://xxiine rsc, ptt.:/TT=DDDDDDDD/o
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction Tree Variational Autoencoder for Molecular Graph Generation, March 2019. URL huthhttp:... // arxiv. wwtp:////////////////// 04364 wwwp:1802244450 [cs, stat].

---

Published as a conference paper at ICLR 2023
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical Generation of Molecular Graphs using Structural Motifs, April 2020. arXiv:2002.03230 [es, stat].
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael L. Townshend, and Ron Dror: Learning from Protein Structure with Geometric Vector Perceptrons, May 2021. URL http:://aaax
exg/aba/2009. 01411 arXiv:2009.01411 [cs, q-bio, stat].
Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional Dif- fusion for Molecular Conformer Generation, June 2022. URL http::/arxix wttpxbbvorgggs////o
0206601729 arXiv:2206.01729 [physics, q-bio].
Greg Landrum. RDKit: Open-source cheminformatics. 2010. URL https://www. www.t....kktt... org/
Yibo Li, Jianfeng Pei, and Luhua Lai. Learning to design drug-like molecules in three-dimensional space using deep generative models. Chemical Science,
2021. BRLhttp:
//arxiv.org////// 21 04. 08474 arXiv:2104.00 08474 [es, q-bio].
Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascamu, and Peter Battaglia. Learning Deep Gen- erative Models of Graphs, March 2018. URL http:// htttp://arxiv.og//// wttp:///gg/a///..oooooo 03324 arXiv:1803.03320 [cs, stat].
Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3D Molecules for Target Protein Binding, May 2022. URL arXiv:2204.09410 [es, q-bio].
www.oorgg/abbb.aaee 09410
Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander Gaunt. Constrained Graph Vari- ational Autoencoders for Molecule Design. In Advances in Neural Information Processing Sys- tems, volume 31, 2018.
Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3D Generative Model for Structure-Based Drug Design. January 2022. URL htttps://ppenreiiew..cco wtwp:/nee//ooommm.oy/om/mm odRo
Youzhi Luo and Shuiwang Ji. An Autoregressive Flow Model for 3D Molecular Geometry Generation from Scratch. March 2022. URL
N03Ajc-N55W
Krzysztof Maziarz, Henry Jackson-Flux, Pashmina Cameron, Finton Sirockin, Nadine Schneider, Nikolaus Stiefl, Marwin Segler, and Marc Brockschmidt. Learning to Extend Molecular Scaf- folds with Structural Motifs, April 2022. URL http:: http://awww..oogga//////a/////// wot./oog//////o//oo 03864 arXiv:2103.03864 [cs, q-bio].
Joshua Meyers, Benedek Fabian, and Nathan Brown. De molecular design and generative models. Drug Discovery Today, 26(11):2777-2115, November 2021. URL hts:::// sciencedirect. http://rettt.oom/ecenneeeeeeeeee http:///wc..eeeeeee..ere///////a
Anthony Nicholls, Georgia B. McGaughey, Robert P. Sheridan, Andrew C. Good, Gregory War- ren, Magali Mathieu, Steven W. Muchmore, Scott P. Brown, J. Andrew Grant, James A. Haigh, Neysa Nevins, Ajay N. Jain, and Brian Kelley. Molecular Shape and Medicinal Chemistry: A Perspective. Journal of Medicinal Chemistry, 53(10)3386-33886 May 2010. URL https:
//doi. 1021/ 000.000001mmm0001000000
Mareus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformatiis, 9(1)488, September 2017. URL htRthhhtt:///oii. http://o..oggg... / s 3321 0235-x
Kostas Papadopoulos, Kathryn A. Giblin, Jon Paul Janet, Atanas Patronov, and Ola Engkvist. De novo design with deep generative models based on 3D similarity scoring. Bioorganic & Medici- nal Chemistry, 44, August 2021. URL https: http://wwwwwccccccieeetttttttee. com/ science/
12

---

Published as a conference paper at ICLR 2023
Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets, May 2022. URL http://arxiv.orgg
abs/2205. 07249 arXiv:2205.07249 [es, q-bio].
Marco Podda, Davide Bacciu, and Alessio Micheli. A Deep Generative Model for Fragment- Based Molecule Generation. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, pp. 2250-2250. PMLR, June 2020. URL https:
html
Daniil Polykovskiy, Alexander Zhebrak, Benjamin Sanchez-Lengeling Sergey Golovanov, Oktai Tatanov, Stanislav Belyaev, Rauf Kurbanov, Aleksey Artamonov, Vladimir Aladinskiy, Mark Veselov, Artur Kadurin, Simon Johansson, Hongming Chen, Sergey Nikolenko, Alán Aspuru- Guzik, and Alex Zhavoronkov. Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models. Frontiers in Pharmacology, 11, 2020. URL https://wwel wwww. 2020 565644)
Frontiersing org/ set//articclsseee
/ 3389/fphar Irrontiers..n.orrr/ttt......
Alexander S. Powers, Helen H. Yu, Patricia Suriana, and Ron O. Dror. Fragment- Ligand Gen- eration Guided By Geometric Deep Learning On Protein-Ligand Structure. Biorxiv preprint, Bio- engineering, March 2022.
03.17. 484653
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Learning a Continuous Representation of 3D Molecular Structures with Deep Generative Models, November 2020. URL http:// 08687 arXiv:2010.06687 [cs, q-bio]
arxiv. arxiv.org/abs
Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. Generating 3D molecules condi- tional on receptor binding sites with deep generative models. Chemical Science, 13(9))2701- 2713, 2022. URL http:::/tpppput rSC. org/ en// wwtpt/t//tttttttteoeeeeee wew.iieeeenaninnnnn... 2ndding/2022 SC/ c sc05976a
James P. Roney, Paul Maragakis, Peter Skopp, and David E. Shaw. Generating Realistic 3D Molecules with an Equivariant Conditional Likelihood Model. November 2021. URL https:
//openreview.
Thomas S. Rush. J. Andrew Grant, Lidia Mosyak, and Anthony Nicholls. A Shape-Based 3-D Scaf- fold Hopping Method and Its Application to Bacterial Protein-Protein Interaction. Journal of Medicinal Chemistry, 48(5):1489-144 1495, March 2005. URL http:ttppe//cc..orr/////
jm04016 633o
Victor Garcia Satorras, Emiel Hoogeboom, Fabian B. Fuchs, Ingmar Posner, and Max Welling. E(n) Equivariant Normalizing Flows, January 2022a. URL http::/aaxiv..rg//
09016 6
arXiv:2105.090 09900 [physics, stat].
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) Equivariant Graph Neural Net- works, February 2022b. URL http::/aaxiv..rrt wwtp://wwwwwba.....o//// 09844 arXiv:2102.09844 [cs, stat].
Marwin H. S. Segler, Thierry Kogej, Christian Tyrchan, and Mark Waller. Generating Fo- cused Molecule Libraries for Drug Discovery with Recurrent Neural Networks. ACS Central Science, 4(1):120-131, January 2018. 7bC0512
10. ot1.://ooo.rr.....00///oceeeeet
Gregor Simm, Robert Pinsler, and Jose Miguel Hernandez-Lobato. Reinforcement Learning for Molecular Design Guided by Quantum Mechanies. In Proceedings of the 37th International Conference on Machine Learning, pp. 8959-899999 PMLR, November 2020a. https: //proceedings.. www. press /tt./vllvv//ee st//mmmmm20000 0b. html
Gregor N. C. Simm, Robert Pinsler, Gábor Csányi, and José Miguel Hernández- SayinndeeLLLoaatooSmmmryete Aware Actor- Critic for 3D Molecular Design, November 2020b. URL httl::t/apxii.ortttt
abs/2011 12747 1. [physics, stat].
Martin Simonovsky and Nikos Komodakis. GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders, February 2018 URL http: //tt:i///xxv,... targ/abs/18022 03480 18Xiv:1802.00 032.03480 [es].
13

---

Published as a conference paper at ICLR 2023
Miha Skalic, José Jiménez, Davide Sabbadin, and Gianni De Fabritiis. Shape-Based Based Generative Modeling for de Novo Drug Design. Journal of Chemical Information and Modeling, 59(3): 1205-1214, March 2019. URL https://pusee
org/ 1ttp:////11111.1/////e../.. joim.
8b00706
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation and cunslation-equivariann neural networks for 3D point clouds, May 2018. BRRL:tt://aatt httptp::/axxi...oo / http////www.rr/////////2/22///// 08212 arXiv:1802.08219 [cs].
Javier Vázquez, Manel López, Enric Gibert, Enric Herrro,, and F. Javier Luque. Merging Ligand- Based and Structure-Based Methods Drug Discovery: Overview of Combined Virtual Screening Approaches. Molecules, 25(20) October 2020. Ctthttts:: www. nebi.
nlm. htt.///a....o////////aaaaaaaaaaa
Mingyang Wang, Zhe Wang, Huiyong Sun, Jike Wang, Chao Shen, Gaoqi Weng, Xin Chai, Honglin Li, Dongsheng Cao, and Tingjun Hou. Deep learning approaches for de novo drug design: An overview. Current Opinion in Structural Biology. 72:135- 144, February 2022. URL https:
w/www. www.seeeeeeerrrr..cmm www.co/ccceeeeoooooeeeeeeeeeeeee
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin Solomon. Dynamic Graph CNN for Learning on Point Clouds, June 2019. URL http:
Daaxiv.org/aba/IBBee 07822 arXiv:1801.07829 [cs].
Shuangjia Zheng, Zengrong Lei, Haitao Ai, Hongming Chen, Daiguo Deng, and Yuedong Yang. Deep scaffold hopping with multimodal transformer neural networks. Journal of Cheminformaties, 13(1):87, November 2021. URL https:/ //mpis://dc...o orr./1..11110
-13321-021-0055-3
14

---

Published as a conference paper at ICLR 2023
A
APPENDIX
CONTENTS
Appendix
15
A.I
Overview of definitions, terms, and notations
16
A.2
Scoring rotatable bonds and stereochemistry
18
Random examples of generated 3D molecules
20
Generation statistics)
23
A.5
Shape-constrained molecular optimization
24
A.5.I
Genetic algorithm
24
A.5.2
Visualization of optimized molecules
24
A.6
Comparing simy ROCS scoring function
................................
27
A.7
Exploring different values of a in simg
28
A.8
Heuristic bonding geometries and their impact global shape
................................
29
A.9
Ablating equivariance
A.10 10 Auxiliary training losses
32
A. II Overview Vector Neurons (VN) operations
33
A.12 12 Graph neural networks
34
A. 13 Fragment library.
35
A. 14 Model parameters
36
A.I5 15 Additional training details
40
A. 16 Additional generation details
42
A.17 Relaxation of generated geometries
43
A. 18 Comparison to LigDream (Skalic et al., 2019)
45
15

---

Published as a conference paper at ICLR 2023
A.1
OVERVIEW OF DEFINITIONS, TERMS, AND NOTATIONS
Table 2: List of definitions, terms, and notations
Symbol
Description
M
a 3D molecule
G
2D molecular graph
G
3D coordinates of a molecule with graph G
ra
coordinates atom R
S
a molecular shape
Ms
an encoded 3D molecule with target shape $ generateddsampled 3D molecule
M'
M
an objective-optimin 3D molecule
simg (M' MMs
graph (chemical) similarity between molecules M' and Mg settionally-sensiiiie shape similarity of G and G shape similarity of molecules MA and MB after Ma is optimally aligned to MB (with ROCS)
simg( G GB)
sims M MB)
parameter controlling the width of the atom-centered Gaussians when computing sims (MA, My)
P(M[S)
conditional distribution
molecules given a molecular shape
M
partial 3D) molecule (e.g., during generation)
G
partial molecular graph (e-g., during generation)
Mo
small 3D substructure used to seed generation
M(()
partial 3D molecule after bonding atoms/fragments the focus
G
partial molecular graph after bonding c atoms/fragments to the focus number of heavy atoms
"3
number of sampled points per heavy atom variance of isotropic atom-centered Gaussians in 3R3 the atom/fragment library
E
fi
an atom/fragment in Lj
f
the learned embedding of atom/fragment fi e Cy learned embedding of atom a point cloud representation of shape S an arbitrary rotation matrix in R3x3
h())
Ps
SO(3)
the special orthogonal group (rotations) in 3
SE(3)
the special Euclidean group (rotations and translations) in R3 tensor containing the learned equivariant features of each point in point cloud; X, (x))
X,
x
tensor containing the learned equivariant features of each atom in the en- coded 3D molecule; X E TR X9x3
(Ist) dimensionality of vector features X, X matrix containing the learned atom embeddings; H (h(*)/a E G} invariant shape features of atom (") E Tg
q
H
(()
x
matrix containing the invariant shape features of each atom in the encoded 3D molecule; X E Ra G
da
dimension of input atom features
d
dimension of learned fragment embeddings
dr
dimension of learned atom embeddings
h
vector of means of the posterior distribution N N(g h.)
h
vector standard deviations the posterior distribution N(,,, ha) sampled variational atom features
hyar
A
interpolation factor between N Nhh,,, and (0, 1) multidimensional standard normal distribution with μ 0, -
10,
16

---

Published as a conference paper at ICLR 2023
Table 3: List of definitions, terms, and notations (continued)
Symbol
Description
Hvar
matrix containing the sampled variational atom features for each atom in the encoded molecule
ela)
sampled Gaussian noise vector for atom o; c(0) ((, 1)
Wia)
learned linear transformation for atom a, applied to {(a tensor containing the per-atom equivariant shape features, after mixing with H H E R xd, x3
H
XH
X
(a)
equivariant shape features for atom a, after mixing with h
Har
invariant embedding X (a)
Har
Har
z
global equivariant representation of the encoded molecule global invariant representation of the encoded molecule
z
dz
dimensionality of Rd. x3 and z E Rd.
H',
W H:
X',
analogues to H, WH. X, XH... XH.,, Z, z, but for the encoded par- tially generated molecule
XH.
X H. Z' z'
sum-pooled T(a)) over the atoms currently in focus /(a)
2
sum-pooled XH over the atoms currently in focus
Zdee
equivariant features of the global system (M 1) Ms)
Ziee
invariant features the dimensionality of Zuee the predicted probability of stopping local generation probability threshold to stop local generation
global system (M (M(e-1) Ms)
dosc
Pu
T
Procus
predicted probabilities over the attachment sites on the focus the atom/fragment types in L, predicted probabilities over the attachment sites on the next fragment the atom of attachment on the focus
Pnext
predicted probabilities
Psite
afoc
fnext
the atom//ragment to be added next the graph of the fragment to be added next the atom of attachment on the fragment to be added next rotation angle (dihedral) of the bond connecting the focus to its parent (tree) node in the partially generated molecule query dihedral angle when scoring the rotation angle of the focal rotat- able bond (+/41 vitc))
fwo
agite
14+11
v/too
the 3D molecule Mi after setting the query dihedral v = thtoc
(Pice iec, scorer
analogue Ziee, but for the rotatable bond scorer when encoding the system (Ms, (((6))))
G% L graapb-gen Lo
stereoisomer of the focus Gi
total loss for training graph generator
binary cross entropy loss for predicting to stop local generation cross entropy loss for selecting the attachment site on the focus cross entropy loss for selecting which atom/fragment to add next modified cross entropy loss for selecting the attachment site on the next fragment
T T-fecus
L
Lsite
Lioodd
cross entropy loss for predicting the bond order
LKt.
KL-divergence loss between N N(,,, h,) and N 1)
Shext-shape
auxiliary loss used when predicting the atom/fragment add auxiliary loss used when predicting whether to stop local generation weighting of LKL Lgraph-gen weighting of Lmext-shape in Lgraph-gon weighting of Tag-shape I Tegrphhge gen
Lo-shape
3KL
3 Snext-shape
Fa-shape
17

---

Published as a conference paper at ICLR 2023
P11
No conformers
No
GTroc
(prediually
coordiaatios
Figure 4: Demonstration of conformer sampling when scoring rotatable bonds. For each query dihe- dral Vitce 1/,, 2/2,... [-T,, T). we sample Ne conformations of the subgraph GTa,. (keeping three held fixed) induced by the subtree Tio whose root node is the current focus. For each sampled (sub- graph) conformation GTi.. we compute the shape similarity of between the sampled conformation and the ground truth (subgraph) conformation G We select the maximum computed (non- aligned) shape similarity sim' amongst the No sampled conformations as the regression target for that theee.
A.2 SCORING ROTATABLE BONDS AND STEREOCHEMISTRY
Recall that our goal is train the scorer to predict
max Sim's("'vveen Gs) for various query Piaa.V
dihedrals 4/411 = ehooc- That is, we wish to predict the maximum possible shape similarity of the final molecule M' to Ms when fixing 1/41 = Vive and optimally rotating all the yet-to-be-scored (or generated) rotatable bond dihedrals 4/42 v/+
maximize sim (Pise) Gs).
Training. We train the scorer independently from the graph generator (with a parallel architecture) using mean squared error loss between the predicted scores a (Pee doc, sigmoid(MLPP dea, scorer
(thne)
and the regression targets g (vime) for N₂ different query dihedrals v'roe [-T,,T]. T)):
1 N.
( scorer
Licoter
(14)
Computing regression targets. When training with teacher foreing (Mf Ms. G' Gs), we compute regression targets the max sim' (G" (Win), Gs) by setting the focal dihedral
$242.Vt+3....
4/141 throe sampling No conformations of the "future" graph Gine induced by the subtree Tic max sims Tix Gs ;
whose root (sub)tree-node is the focus, and computing Syie i=
(G
2.0). Since we fix bonding geometries, we need only sample No sets of dihedrals of the rotatable bonds in GS, to sample No conformers, making this conformer enumeration very fast. Note that rather than using sensitive to shape differences (App. A. When computing regression targets, we use N₂ < 1800 and select 36 (evenly spaced) thocus 1-x,, x π per rotatable bond. Figure visualizes how regression targets are computed. App. A.15 contains further training specifics.
0.81 in these regression targets, we use 2.0 to make the scorer more
Scoring stereochemistry. At generation time, also enumerate all possible stereoisomers of the focus (except cis/trans bonds) and score each stercoisomer separately, ultimately selecting the (stereoisomer, v'nse) pair that maximizes the predicted score. Figure FiguureSillussrates illustrates how enumerate stereoisomers. Note that although we use the learned scoring function to score stereoisomerism at generation time, we do not explicitly train the scorer score different stereoisomers.
Masking severe steric clashes. At generation time, we do not score any query dihedral whne that causes steric clash (< 1Å) with the existing partially generated structure (unless all query dihedrals cause severe clash).
18

---

Published as a conference paper at ICLR 2023
Swapping Chiral Atoms
Swapping Axial Equatorial Attachments to Focus Fragment
Swapping Axial Equstorial Attachment of Focus Fragment
Swapping Cis / Trans Bonds [not yet included in SQUID)
OH
OH
.OH
OH
OH
OH
Wio
Weee
OH
focus scoring
Figure 5: addition to scoring the dihedral of the focal bond, we also enumerate all stereoiso- mers around the focus and score each stercoisomer's focal dihedral separately. We then select the best scoring (steroisomer, dihedral) pair. We enumerate stereoisomers by combinatorially swap- ping chiral atoms, swapping axial/equatorial attachments to the foeus (if relevant), and swapping the axial/equatorial attachment of the focus to the rest of the partial molecule (if relevant). Although swapping cis/trans bonds is feasible within SQUID's framework, cis/trans isomers because of their minor presence in the dataset. Hence, any generated double bonds will have random stereochemistry.
currently do not enumerate
19

---

Published as a conference paper at ICLR 2023
A.3 RANDOM EXAMPLES OF GENERATED 3D MOLECULES
Tigures[aandd 7 show additional random examples of molecules generated by SQUID when sampling
Nmax 1, 20 molecules with simg .Ms) 0.7 from the prior (A = 1.0) À 0.3 and selecting the sample with the highest sim [M', Ms). Note that the visualized poses of the generated conformers are those which are directly generated by SQUID; the generated conformers have not been explicitly aligned to Ms (c.g., using ROCS). Even so, the conformers are (for the most part) aligned to My since SQUID's equivariance enables the model to generate natively aligned structures.
It is apparent in these examples that using larger Nma yields molecules with significantly improved shape similarity to Ms. both qualitatively and quantitatively. This is in part caused by: 1) stochas- ticity in the variationally sampled atom embeddings Hvari 2) stochasticity in the input molecular point clouds, which are sampled from atom-centered isotropic Gaussians R$; 3) sampling sets of variational atom embeddings that may not be entirely self-consistent (e.g., for instance, if we sample only atom embedding that implicitly encodes ring structure); and 4) the choice of Tg. the thresh- old for stopping local generation. While small тΦ (we use тΦ 0.01) helps prevent the model from adding too many atoms or fragments around a single focus, a small Ty can also lead to early (local) stoppage, yielding molecules that do not completely fill the target shape. By sampling more molecules (using larger Nmax), we have more chances to avoid these adverse random effects. Further work will attempt to improve the robustness of the encoding scheme and generation procedure in order to increase SQUID's 's overall sample efficiency.
20

---

Published as a conference paper at ICLR 2023
Ms
= 0.3
== 1.0 (prior)
>= 0.3
A 1.0 (prior)
sima(M,, (M))
sMma Ms]
simg(M", Ma)
simg(M [M)
Nmax
Nmac
Nmax 20
Nmax 20
a
arta
foo
Figure 6: Random examples of molecules generated by SQUID using different sampling strategies.
The generated 3D molecules are shown overlaid on the target molecule Ms (blue). these exam- ples, the generated molecules have not been explicitly aligned to Ms. and their displayed poses those directly generated by SQUID.
21

---

Published as a conference paper at ICLR 2023
À
== 1.0 (prior)
0.3
À 1.0 (prior)
Ms
simg Ma) <, 0.7
simg(M', M <
sima (M', Mg) < 0.7
simg(M', < 0.7
Nmax
Nmax
Nmax 20
Nmxx 20
Figure 7: Additional random examples of molecules generated by SQUID using different sampling strategies. The generated molecules are shown overlaid on target molecule Mş (blue). In these examples, the generated molecules have not been explicitly aligned to Ms, and their displayed poses are those directly generated by SQUID.
22

---

Published as a conference paper at ICLR 2023
A.4 GENERATION STATISTICS
reports the percentage of molecules that are chemically valid, novel, and unique when sam- pling 50 molecules from the prior (A 1.0) for 1000 encoded molecules My (e.g., target shapes) from the test set, yielding a total of 50K generated molecules. We define chemical validity as pass- ing RDKit sanitization. Since we directly generate the molecular graph and mask actions which violate chemical valency, 100% of generated molecules are valid. We define novelty as the percent- age of generated molecules whose molecular graphs are not present in the training data. We define uniqueness as the percentage of generated molecular graphs (of the 50K total) that are only gen- erated once. For novelty and uniqueness calculations, we consider different stereoisomers to have the same molecular graph. We also report the percentage of generated 3D structures that have an apparent steric clash, defined to non non-bonded interatomic distance below 2Å.
When sampling from the prior (A 1.0), the average internal chemical similarity of the generated molecules is 0.26 = 0.04. When sampling with 0.3, the average internal chemical similarity is 0.32 = 0.07. We define internal chemical similarity to be the average pairwise chemical similarity (Tanimoto fingerprint similarity) between molecules that are generated for the same target shape.
Table 5 reports the graph reconstruction accuracy when sampling 3D molecules from the posterior (A -A= 0.0). for 1000 target molecules Mg from the test set. We report the top-k graph reconstruc- tion accuracy (ignoring stereochemical differences) when sampling k molecule per encoded Ms. and when sampling = 20 molecules per encoded Ms. Since we have intentionally trained SQUID inside shape-conditioned variational autoencoder framework in order generate chemi- cally diverse molecules with similar 3D shapes, the significance of graph reconstruction accuracy debatable in our setting. However, it is worth noting that the top-1 reconstruction accuracy is 16.3%, while the top-20 reconstruction accuracy is much higher (57.2%). This large difference is likely attributable both stochasticity in the variational atom embeddings and stochasticity in the input 3D point clouds.
Table 4: Generation statisties computed by sampling 50 molecules from the prior (A 1.0) per encoded target molecule Mg for 1000 random targets from the test set.
Statistic
%
Chemical Validity (t)
100%
Novelty (1)
98.8%
Uniqueness (1)
94.7%
Steric Clash (<2A) (L)
13.1%
Table 5: Graph reconstruction accuracy when sampling 3D molecules from the posterior (A = 0.0) for 1000 targets Ms from the test set. We report the "top-k" reconstruction accuracy when sampling k = molecule per target, as well as k 20 molecules per target.
Graph Reconstruction
%
16.3%
A = 20
57.2%
23

---

Published as a conference paper at ICLR 2023
A.5 SHAPE-CONSTRAINED MOLECULAR OPTIMIZATION
A.5.1 GENETIC ALGORITIM
We adapt SQUID to shape-constrained molecular optimization by implementing genetic algorithm on the variational atom embeddings H Algorithm detaail the exact optimization procedure. In summary, given the seed molecule Ms with a target 3D shape and an initial substructure Mo (which contained by all generated molecules for given Ms). we first generate an initial population of generated molecules MM' by repeatedly sampling Hvar for various interpolation factors X, mixing these Hvar with the encoded shape features Ms. and decoding new 3D molecules. We only add generated molecule to the population if simy (M' Ms) Ts (we use Tş 0.75), so that the GA does not overly explore regions of chemical space that have no chance of satisfying the ultimate constraint (MM", .Mg)= 2 0.85. After generating the initial population, we iteratively 1) select the lop-scoring samples in the population, 2) cross the top-scoring Hvar in crossover events, 3) mutate the top and crossed Hvar via adding random noise, and 4) generate new molecules M' for each mutated H The final optimized molecule M is the top-scoring generated molecule that satisfies the shape- saape--imllattt constraint sims Ms) 2 0.85.
A.5.2 VISUALIZATION OF OPTIMIZED MOLECULES
Figure visualizes the structures of the SQUID-optimized molecules M and their respective seed molecules Ms (e.g., the starting "hit" molecules with target shapes) for each of the optimization tasks which led an improvement in the objective score. We also overlay the generated 3D confor- mations of M on those of Ms. and report the objective scores for each M and Mg.
24

---

Published as a conference paper at ICLR 2023
Algorithm Genetic algorithm for
optimization with SQUID
Given: Ms with n heavy atoms, My, objective oracle O Params: 7g, TG, No: Nr. Ne Defaulls: тς 0.75, TO 0.95, N₂ 20, Ny 20, Nv. 10
H., H Encode (Mss
Encode target molecule
Initialize population P = {[Ms, Hp))
for [0.0, 0.2, 0.4, 0.6, 0.8, 1.0] do for i 1, ..... 100 do
P Create initial population of (M', H
Sample noise c E R (n xdu) (()
X1) Mutate variational atom embeddings Har (1 - A) t ((1 H)Ho
z, Encode(Ms; Hyar)
P Mix mutated chemical and shape information
M' = Decode(,M, (M, z, Z.)
4 Generate mutated molecule M'
Compute (M', Ms)
if sim (M', Mg) >= Ts then
Add M' to population only simg (M', MMg) is high
Add (M' H to P
end if
end for end for
for = I,.., No do
D For each evolution Sort population by objective score, high to low.
Construct P soned by sorting by O(M) Initialize TM = {}. T.. = 0
for (M, H
sooted do
Collect top-Nr scoring (M', H.ar)
if (simg MT) < <G 4 Mr E TM)) and (I7) Nr) then
Add M to TM
Add Hvar to T....
end if
end for
Initialize To = =0 for e= 1,.... No do
Add crossovers to set of top-scoring H
Sample H, H.. H 1 H. = CROSS(H, H2)
Th > Cross by randomly swapping half of the atom embeddings
Add H. to Tc
end for Th... = Th. UTc
for H E TT.,, do
Adding to population
for À E [0.0, 0.2, 0.4, 0.6, 0.8, 1.0] do for i I..., 10 do
Sample noise E (n (O,
H (1 H) +e
Mutate variational atom embeddings > Mix mutated chemical and shape information
z, Z Encode(Me, Ms; Hymr)
M' Decode z, Z)
> Generate mutated molecule M'
Compute sims (M'' .Ms)
if sims(M' MMs) >= Ts then
D Add M' to P only if sims Ms) is high
Add (M'' H to
end if
end for
end for
end for
end for
return M = arg max O(M') subject to sims(M', (M' >= -.85
MrE
25

---

Published as a conference paper at ICLR 2023
Seed:
H
GSK3B
Soad:
JWKa
Seed:
Osimertinis MPO
Seed:
Sitegliptin MPO
0.4
032
Seed:
Colecouts Rodiscovery
-00 Rooo
Seed:
Thiohhoenee Rediccovory
Figure 8: Results of shape-constrained MO for the 40/48 tasks which improved the objective score. Subpanels depict the target molecules Ms (top row), the row), the overlaid 3D structures (bottom row; blue is Ms. red is M ). and their respective objective scores for the (from top to bottom) GSK3B, JNK3, Osimertinib MPO, Sitagliptin MPO, Celecoxib Rediscovery, and Thiothixene Rediscovery tasks. In the overlaid structures, the generated molecules have nor been explicitly aligned to Mg; the poses shown are those directly generated by SQUID.
molecules A (middle
26

---

Published as a conference paper at ICLR 2023
1.0
ROCS Computed Shape Scores
0.9
0.8
0.7
0.6
0.5
0.4
0.3
MAE: 0.0016
0.2
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Our Computed Shape Scores
Figure 9: Parity plot empirically showing the close approximation of Eq. (a 0.81) to the shape-similarity function used by the commercial ROCS program, for 50000 shape comparisons.
A.6 COMPARING SIMs TO ROCS SCORING FUNCTION
Our shape similarity function described in Equation D closely approximates the shape (only) scoring function employed by ROCS, when a = 0.81. Figure 9 demonstrates the near pear-perfect correlation between computed shape scores and those computed by ROCS for 50,000 shape comparisons, with a mean absolute error of 0.0016. Note that Equation computes non-aligned shape similar- ity. We still employ ROCS to align the generated molecules M' to the target molecule Ms before computing their (aligned) shape similarity in our experiments. However, we do not require explicit alignment when training SQUID; we do not use the commercial ROCS program during training.
27

---

Published as a conference paper at ICLR 2023
1.0
dataset (N
0.9
dataset (Na. 20)
Mean Shape Similarity
dataset (Nas 500)
0.8
À - 0.3 (Mmas 1)
= 1.0 (Wmax 1)
A 0.3 (Wmas 20)
À 1.0 (Wmax 20)
0.6
0.5
0.4
0.2
0.81
1.20
1.60
2.00
2.40
2.80
3.20
3.60
4.00
α
Figure 10: Mean shape similarity simg (M: for increasing values of a (Equation 1) for 1000 target molecules Ms from the test set. When sampling either from the dataset from SQUID, compute the mean shape similarity (across the 1000 targets) after selecting the best sample M which maximizes sims (M', Mg;a) amongst Nmax samples M' for which simg (M', Mg) 0.7.
A.7 EXPLORING DIFFERENT VALUES OF IN SIMs
Our analysis of shape similarity thus far has used Equation I with a = 0.81 in order to recapitulate the shape similarity function used by ROCS, which is widely used in drug discovery. However, compared to randomly sampled molecules in the dataset, the molecules generated by SQUID quali- tatively appear do a significantly better job fitting the target shape on atom-by-atom basis, even if the computed shape similarities (with a = 0.81) are comparable (see examples in Figure We quantify this observation by increasing the value of when computing sims Mg; a) for generated molecules M', ,', as a is inversely related to the width of the isotropic 3D Gaussians used in the volume overlap calculations in Equation 1, Intuitively, increasing will greater penalize sims if the atoms of M' and Ms do not perfectly align.
Figure plots the mean (M, Mg; a)) for the most shape-simille molecule M of Nmax sampled molecules M' for increasing values of a. Averages are calculated over 1000 target molecules Ms from the test set, and we only consider generated molecules for which simg M' Ms) < 0.7. Cru- cially, the gap between the mean sims (M, Ms; a:) obtained by generating molecules with SQUID vs. randomly sampling molecules from the dataset significantly widens with increasing a. This effect is especially apparent when using SQUID with X = 0.3 and N 20, although can be ob- served with other generation strategies as well. Hence, SQUID does a much better job at generating (still chemically diverse) molecules that have significant atom-to-atom overlap with Mg.
28

---

Published as a conference paper at ICLR 2023
A.8 HEURISTIC BONDING GEOMETRIES AND THEIR IMPACT ON GLOBAL SHAPE
In all molecules (dataset and generated) considered in this work, we fix acyclic bond distances to their empirical averages and set acyclic bond angles to heuristic values based on hybridization rules in order to reduce the degrees of freedom in 3D coordinate generation. Here, we describe how we fix these bonding geometries and explore whether this local 3D structure manipulation significantly alters the global molecular shape.
Fixing bonding geometries. We fix acyclic bond distances by computing the mean bond distance between pairs of atom types across all the RDKit-generated conformers in our training set. After collecting these empirical mean values, we manually set each acyclic bond distance its respective mean value for each conformer in our datasets.
We set acyclic bond angles using simple hybridization rules. Specifically, sp3-hybridized atoms will have bond angles of 109.5%, sp2-hybridized atoms will have bond angles of 120", and sp-hybridized atoms will have bond angles of 180%. We manually fix the acyclic bond angles to these heuristic values for all conformers in our datasets. We use RDKit to determine the hybridization states of each atom. During generation, occasionally the hybridization of certain atoms (N, O) may change once they are bonded to neighbors. For instance, sp3 nitrogen can become sp2 once bonded to an aromatic ring. We adjust bond angles on-the-fly in these edge cases.
Impact global shape. Figure I1 plots the histogram of sims( Mrelused) for 1000 test set conformers Mixed whose bonding geometries have been fixed, and the original RDKit- generated conformers Mrelaxed with relaxed (true) bonding geometries. In the vast majority of cases, fixing the bonding geometries negligibly impacts the global shape of the 3D molecule (sims( (Mixed, Mrelaxdd) 1). This is because the main factor influencing global molecular shape is rotatable bonds (e.g., flexible dihedrals), which are not altered by fixing bond distances and an- gles.
Recovering refined bonding geometries. Even though fixing bond distances and angles only marginally impacts molecular shape, we still may wish to recover refined bonding geometries of the generated 3D molecules without altering the generated 3D shape. We can accomplish this (to a first approximation) for generated molecules by creating a geometrically relaxed conformation of the generated molecular graph with RDKit, and then manually setting the dihedrals of the ro- tatable bonds in the relaxed conformer to match the corresponding dihedrals in the generated con- formers. Importantly, if we perform this relaxation procedure for both the dataset molecules and the SQUID- -euD--eeretted molecules, the (relaxed) generated molecules still have significantly enriched shape-similarity to the (relaxed) target shape compared (relaxed) random molecules from the dataset (Fig. 12
29

---

Published as a conference paper at ICLR 2023
400
Frequency
300
200
0
0.825
0.850
0.875
0.900
0.925
0.950
0.975
1.000
Shape Similarity
Figure 11: Histogram of shape similarity (Mnxed Myelssed) between 1000 test molecules whose acylic bonding geometries are fixed by heuristies and their geometrically relaxed counterparts.
140
dataset
(prior)
Frequency
80
60
40
20
0
0.4
0.5
0.6
0.7
0.8
0.9
Shape Similarity
Figure 12: Distribution of sim'(M', .ss) when sampling (Nmax = 20, simg (M' Ms) 0.7)
molecules M' from SQUID (A 1.0) and from the dataset, when relaxing the bonding geometries of each M' and Ms. Importantly, SQUID generates 3D molecules that are significantly enriched in shape similarity even after relaxing the (fixed) bonding geometries of the generated structures.
30

---

Published as a conference paper at ICLR 2023
Shape Similarity
0.8
0.5
Delaset
0.4
SOLID
EQUID (k-00)
NoEqua (3 00g
Dataset
SQUID-koFqui sim Now
SQUID
SQUID-NoEqi
SQUIDD sim,
Dataset simy =300
SQUID-AnEqul sin, <00 Noes
SOUID
SQUID-NoFgus samy <
SQUID sim, <00
s10%, 0.7
sin <0.
Mow
Sampling Scheme
Figure 13: Distributions of sims(M', Ms) for the best of Nmax generated molecules for which sima M', , Ms) < 0.7 or 0.3, when using SQUID or SQUID-NoEqui. Generated distributions are compared those obtained by similarly sampling molecules from the training dataset. Overall, ablating SQUID's equivariance signficantly reduces the enrichment in shape similarity of generated molecules the target shape.
A.9 ABLATING EQUIVARIANCE
SQUID aligns the equivariant representations of the encoded target shape and the partially generated structures in order generate 3D conformations that natively fit the target shape, without having to implicitly learn SE(3) -Elgnmaliss (Challenge 2). We achieve this in Equation / where we mix the equivariant representations of Ms and the partially generated structure M, M(c-1) To empirically motivate this design choice, we ablate the equivariant alignment by setting 2 = 0 in Eq. We de- note this ablated model as SQUID-NoEape features z to the decoder (Eq. 8), SQUID-NoEqui is still conditioned on the shape of Ms - the model simply longer has access to any explicit information about the relative spatial orientation of M Mg (and thus must learn this spatial relationship from scratch). As expected, ab- lating SQUID's equivariance significantly reduces SQUID's s ability to generate chemically diverse molecules that fit the target shape.
Note that because we still pass the unablated invariant
Figure 13 plots the distributions of S for the best of Nmax generated molecules with simg M MM) < 0.7 or 0.3 when using SQUID or SQUID-NoEqui. Crucially, the mean shape similarity when sampling with (A 1.0, Nmux 20, sima(M', .Ms) 0.7) decreases from 0.828 (SQUID) to 0.805 (SQUID-NoEqui When sampling with (A = 0.3, Nmax = 20, simg Ms) < 0.839 (SQUID-NOEEqii)
0.7), the Relative to the mean shape similarity of 0.758 achieved by sampling random molecules from the dataset (Nmaxx 20, simg (M', .Ms) 0.7), this corresponds to a substantial 33% reduction in the shape-enrichment of SQUID- molecules.
shape similarity also decreases from 0.879 (SQUID)
Interestingly, sampling (A 1.0, Nmax 20, sima(M) Ms) 0.7) with SQUID-NoEqui still yields shape-enriched molecules compared to analogously sampling random molecules from the dataset (mean shape similarity of 0.805 vs. 0.758). This is because even without the equivariant feature alignment, SQUID-NoEqui still conditions molecular generation on the (invariant) encoding of the target shape S, and hence biases generation towards molecules which better the target shape (after alignment with ROCS).
31

---

Published as a conference paper at ICLR 2023
A.10 AUXILIARY TRAINING LOSSES
We employ two auxiliary losses when training the graph generator in order to encourage the gener- ated graphs to better fit the encoded target shape.
The first auxiliary loss penalizes the graph generator if it adds an incorrect atom/fragment to the focus that is of significantly different size than the correct (ground truth) atom//ragment. We first compute a matrix AV, E ([((()))())) containing the (pairwise) volume difference between all atoms/fragments in the library Ly
AVI. (i)) |ug -
(15)
where vy, is the volume atom/fragment E Ly (computed with RDKit).
We then compute the auxiliary loss Leext-shape as:
Leext-huppe
y (Peext
A AV())
(16)
where g is the index of the correct (ground truth) next atom/fragment fnext, true. A AVIa is the gth row of AV,, f, and Pnext are the predicted probabilities over the next atom/fragment types to be connected to the focus (see Eq. (O).
The second auxiliary loss penalizes the graph generator it prematurely stops (local) generation, with larger penalties if the premature stop would result in larger portions of the (ground truth) graph not being generated. When predicting (local) stop tokens during graph generation (with teacher forcing), we compute the number of atoms in the subgraph induced by the subtree whose root tree- node is the next atom/fragment to be added to the focus (in the current generation sequence). We then multiply the predicted probability for the local stop token by this number of "future" atoms that would not be generated if premature stop token were generated. Hence, if the correct action is to indeed stop generation around the focus, the penalty will be zero. However, if the correct action is to add a large fragment to the current focus but the generator predicts stop token, the penalty will be large. Formally, we compute:
Lp-shupe Pe Gs if Pa, tre
otherwise
(17)
where Pa, tee is the ground truth action for local stopping (Pa, tne 0 indicates that the correct action is to not stop local generation), and Gs the subgraph induced by the subtree whose root node the next atom/fragment (to be generated) in the ground-truth molecular graph.
32

---

Published as a conference paper at ICLR 2023
A.11 OVERVIEW OF VECTOR NEURONS (VN) OPERATIONS
In this work, we use Deng et al. 2021)'s 's VN-DGCNN to encode molecular point clouds into equiv- ariant shape features. We also employ their general VN operations (VN-MLP, VN-Inv) during shape and chemical feature mixing. We refer readers to Deng et al. 2 for a detailed description of these equivariant operations and models. Here, we briefly summarize some relevant VN-operations for the reader's s convenience.
VN-MLP. Vector neurons (VN) lift scalar neuron features to vector features in R Hence, instead of having features x R9, we have vector features X Rax3, While linear transformations naturally equivariant to global rotations R. since W(XR) = (WX)R for some rotation matrix R E 3x33 Deng al. (202)) construct set of non-linear equivariant operations f such that f(XR) RR thereby enabling natively equivariant network design.
VN-MLPs combine linear transformations with equivariant activations. In this work, we use VN- Lea which Deng al. (202T) define as:
VN-LeakyReLU((X.a) a (11-a)VN-ReLU(X
(18)
where
x
if k
0
VN-ReLU(X))
k
VXEX
(19)
x
otherwise
[a]]
where k = UX for learnable weight matrix U E R 1x and where & R3
By composing series of linear transformations and equivariant activations, VN-MLPs map X Rox3 X' T such that X'R VN-MLP(XR).
VN-Inv. Deng al. 2021 also define learnable operations that map equivariant features X R2x3
to invariant features x E 3 In general, VN-Inv constructs invariant features by multiplying equivariant features X with other equivariant features Y E 3x33
XY
(20)
The invariant features X E R can then be reshaped into standard invariant features x E 3 In our work, we slightly modify [eng et al. (202T)'s s original formulation. Given a set of equivariant features X {X0) "x we define a VN-Inv as:
VN X =X
(21)
where X = (xx(0 E Rax6g and:
x Flatten F (
(22)
VO=
Σ ()) if otherwise (
(23)
T. = (N-MLP(V(0)
(24)
where T. E R3x3 and (0) E 22xx3 (n > 1) or Vi E R (n=
VN-DGCNN, Deng al. 2021 introduce VN-DGCNN
SO(3) -Quii-equi version of the
Dynamic Graph Convolutional Neural Network (Wang et al. 2019). Given a point cloud P E Rax3
VN-DGCNN uses (dynamic) equivariant edge convolutions update equivariant per-point features:
(1+1)) VN-LeakyReLU
xx()) & (0)) x())
(25)
(++1)
((++))
(26)
mm
mEKNN, (x)
where KNN (n) are the k-nearest neighbors of point m in feature space, (()) and (t) are weight matrices, and X (1) IRxx3 are the per-point equivariant features.
33

---

Published as a conference paper at ICLR 2023
A.12 GRAPH NEURAL NETWORKS
In this work, we employ graph neural networks (GNNs) to encode:
each atom/fragment in the library L
the target molecule Ms
each partial molecular structure M((() during sequential graph generation
the query structures ((()))) when scoring rotatable bonds
Our GNNs are loosely based upon a simple version of the EGNN (Satorras et al. 2022b). Given
molecular graph G with atoms nodes and bonds as edges, graph convolutional layers defined by the following:
m
6%
h hy. ||,; - r,]1?, m!:
(27)
m ++1
Σ mij
(28)
JEN()
h [==1) 0% (0) (h. m (t=1))
(29)
h 0f (h', m +1)
he (t > 0)
(30)
where hf are the learned atom embeddings at each GNN-layer, m learned (directed) messages, ry E 3 the coordinates of atom i, [N) is the set of 1-hop bonded neighbors of atom and each 6t of, are MLPs. Note that h. are the initial atom features, and m. are the initial bond features for the bond between atoms and j. In general, mi, o m/j for t 0, but here m. m Note that since we only aggregate messages from directly bonded neighbors, ||r; ryll only encodes bond distances, and does not encode any information about specific 3D conformations. Hence, our GNNs effectively only encode 2D chemical identity, as opposed to 3D shape.
34

---

Published as a conference paper at ICLR 2023
Figure 14: All 100 distinct fragments in atom/fragment library Ly-
A.13 FRAGMENT LIBRARY.
Our atom/fragment library Cy includes 100 distinct fragments (Fig. 14] and 24 unique atom types. The 100 fragments were selected based the top- 100 most frequently occurring fragments in our training set. In this work, we specify fragments as ring- aingccotaining substructures that do not con- tain any acyelic single bonds. However, in principle fragments could be any (valid) chemical sub- structure. Note that we only use (geometrically optimized) conformation per fragment, which is assumed be rigid. Hence, in its current implementation, SQUID does not consider different ring conformations (c.g., boat vs. chair conformations of cyclohexane).
35

---

Published as a conference paper at ICLR 2023
A.14 MODEL PARAMETERS
Parameter sharing. For both the graph generator and the rotatable bond score, the (variational) molecule encoder (in the Encoder, Fig. 2) and the partial molecule encoder (in the Decoder, Fig. 2) share the same fragment encoder (CN-,NN), which is trained end-to-end with the rest of the model. Apart from Ly-GNN, these encoders do not share any learnable parameters, despite having parallel architectures. The graph generator and the rotatable bond scorer are completely independent, and are trained separately.
Plypepparametes. Tables 6 and tabulate the set of hyperparameters used for SQUID across all the experiments conducted this paper. Table@summarrizs training and generation parameters, but we refer the reader to App. A.15 and A.16 for more detailed discussion of training and generation protocols.
Because of the large hyperparameter search space and long training times, we did not perform exten- sive hyperparameter optimizations. We manually tuned the learning rates and schedulers to maintain training stability, and we maxed-out batch sizes given memory constraints. We set f0 90-supe 10 and Suext-shupe = 10 to make the magnitudes of Syesh and Lnea-shppe comparable to the other loss com- ponents for coah-gennraatio. We slowly increase 5KL over the course of training from 10 to maximum of 10 which we found to provide reasonable balance between LK and graph reconstruction.
36

---

Published as a conference paper at ICLR 2023
Table 6: Architecture hyperparameters for SQUID's graph generator. Many of the GNN, VN- DGCNN, Vn-Inv, VN-MLP, and MLP modules use the same set of hyperparameters (not learnable parameters), but some do not. Where appears, (-+) indicates the outputs of the module(s) which we refer (see Fig. 2). The graph generator has a total of 1468103 learnable parameters.
Symbol
Parameter Description
Value
of
variance atom-centered Gaussians
0.049 A
number sampled points per atom
dy
dimensionality of atom/fragment embeddings X E R4x3
64
q
64
da
dimensionality of input atom/node features dimensionality of input bond/edge features dimensionality of atom embeddings
45
db
5
dn
64
q'
W B&' Xq
32
dz
dimensionality of dimensionality of Zaec
64
ditec
448
(GNNc,) [hidden, (output)] layer sizes
[64, (64)]
(GNN c, [hidden, (output)) layer sizes
[64, (64)]
Number of GNN,, layers
Pm
(t=0)
(Enc., Dec. GNNs) [hidden, (output)) layer sizes
[128, (64)]
(Enc., Dec. GNNs) [hidden. (output)) layer sizes
[128, (64)]
(1>00)
(Enc., Dec. GNNs) [hidden, (output)) layer sizes
[64, (64))
Pm
(1>0) (Ene., Dec. GNNs) [hidden, (output)) layer sizes
[64, (64))
Number of (Enc., Dec.j-GNN layers
3
Conv. dimensions for VN-DGCNN (- X, X')
[32, 32, 64, 128)
Conv. pooling type for VN-DGCNN (- X,X'
mean
Number of k-NN for VN-DGCNN (+ (+XXX)
VN-Inv (+ X, X' XH.., X(H) [hidden, (output)] layer sizes
[64, 32, (3) [(3)]
VN-Inv (- Ziee)) [hidden, (output)) layer sizes
VN-MLP XH..: :H) [hidden, (output)) layer sizes
[(64)]
VN-MLP Zitee [hidden, (output)) layer sizes
[128, 64, (64))
MLP (->
(H
pr H))) [hidden, (output)) layer sizes
[64, (128)]
MLP
WH, WH [hidden, (output)] layer sizes
[64, (2048))
MLP
XH. XH) [hidden, (output) layer sizes
[128, 64, (64))
MLP
Pg: Piocus: Prext, Psite: Pbona) [hidden] layer sizes
[64, 64, 64]
MLP
(all) hidden layer activation function
LeakyReLL(0.2)
37

---

Published as a conference paper at ICLR 2023
Table 7: Architecture hyperparameters for SQUID's rotatable bond scorer. Many of the GNN, VN- DGCNN, Vn-Inv, VN-MLP, and MI.P modules use the same set of hyperparameters (not learnable parameters), but some do not. Where appears, (-+) indicates the outputs of the module(s) to which we refer (see Fig. 2). The rotatable bond scorer has a total 1270080 learnable parameters.
Symbol
Parameter description
Value
of
variance of atom-centered Gaussians
0.049 Á
ny
number of sampled points per atom
5
dz
dimensionality of atom/fragment embeddings X E R
64
q
64
da
dimensionality of input atom/node features dimensionality of input bond/edge features dimensionality of atom embeddings W
45
dy
5
dr
64
q'
H E Rd' Xq
32
dimensionality of
64
duec
dimensionality of Zdec, scurer
448
Om (GNNc,) [hidden, (output)] layer sizes
[64, (64)]
(GNN [hidden, (output)) layer sizes
[64, (64))
Number of GNN layers
Pm
(t=0)
(Enc./Dec. GNNs) [hidden, (output)) layer sizes
[128, (64))
(t=0)
(Enc./Dec. GNNs) [hidden, (output)] layer sizes
[128, (64))
Om
(t>0)
(Enc./Dec. GNNs) [hidden, (output)) layer sizes
[64, (64))
(1>0)
(Enc./Dec. GNNs) [hidden, (output)) layer sizes
[64, (64)
h
Number of (Enc./Dec.) D.t./DDe...-GNNNNN layers
3
Conv. dimensions for VN-DGCNN (+ X, X')
[32, 32, 64, 128]
Conv. pooling type for VN-DGCNN (+ X,
mean
Number of k-NN for VN-DGCNN (+ X.X')
10
VN-Inv (++ X, XH. X/H) [hidden, (output)) layer sizes
[64, 32, (3)] [(3)]
VN-InV (+ Zeee) [hidden, (output)) layer sizes
VN-MLP (- X X'H) [hidden, (output)) layer sizes
[(64)]
VN-MLP (+ Zite) [hidden, (output)] layer sizes
[128, 64, (64))
MLP (- WH. W' [hidden, (output)] layer sizes
[64, (2048))
MLP ++ XH, X [hidden, (output) layer sizes
[128. 64, (64)]
MLP (scoring) [hidden, (output)] layer sizes
[64, 64, 64, (1)
MLPP (all) hidden layer activation function
LeakyReI.U(0.2)
38

---

Published as a conference paper at ICLR 2023
Table 8: Summary of training and generation parameters, See App.
A.15
A.16 for further descrip-
tions on training and generation protocols.
Symbol
Parameter description
Value
Training parameters for graph generator
Saus Shext-shape
weighting of sh in loss
10.0
weighting of Lnext-shape in loss weighting of LKL in loss
10.0
3KL
log-linear from 10-5 to 10 over 1M batches
max. learning rate min. learning rate learning rate scheduler
2.5 x 10-4
5 x 10-6
exp. decay (7 0.9) ev-
ery 50K batches
max. batch size (generation sequences) # training iterations (batches)
400
2M
Training parameters for rotatable bond scorer max. learning rate
5 x
min. learning rate
1
10-5
learning rate scheduler
exp. decay (? = 0.9) ev-
ery 50K batches
max. # of focus bonds per batch
32
# of query dihedrals throe per focus bond, per batch # training iterations (batches)
10
2M
Generation parameters
TD
threshold for stopping local generation
0.01
Number of scored dihedrals tho per focus
36
39

---

Published as a conference paper at ICLR 2023
A.15 ADDITIONAL TRAINING DETAILS
Dataset. We use molecules from MOSES (Polykovskiy et al. 2020) train, validate, and test SQUID. Starting from the train/test sets provided by MOSES, we first generate an RDKit conformer for each molecule, and remove any molecules for which we cannot generate a conformer. Con- formers are initially created with the ETKDG algorithm in RDKit, and then separately optimized for 200 iterations with the MMFF force field. We then fix the acyelic bond distances and bond an- gles for each conformer (App. A Using the molecules from MOSES's s train set, we then create the fragment library by extracting the top- most frequently occurring fragments (ring-containing substructures without acylic bonds). We separately generate 3D conformer for each distinct frag- ment, optimizing the fragment structures with MMFF for 1000 steps. Given these 100 fragments, then remove all molecules from the train and test sets containing con-included fragments. From the filtered training set, we then extract 24 unique atom types, which we add to the atom/fragment library L.. We remove any molecule in the test set that contains atom type not included in these 24. Finally, we randomly split the (filtered) training set into separate colning/validatiin splits. The training split contains 1058352 molecules, the validation split contains 264589 molecules, and the test set contains 146883 molecules. Each molecule has one conformer.
Collecting training data for graph generation and scoring. We individually supervise each step of autoregressive graph generation and use teacher forcing. We collect the ground-trui generation actions by representing each molecular graph as a tree whose root tree-node either a terminal atom or terminal fragment in the graph. A "terminal" atom is only bonded to one neighboring atom. A "terminal" fragment has only one acyelic (rotatable) bond to a neighboring atom/fragment. Starting from this terminal atom/fragment, we construct the molecule according to a breadth-first. search traversal of the generation tree (see Fig. 2); we break ties using RDKit's canonical atom ordering. We augment the data by enumerating all generation trees starting from each possible terminal atom/fragment in the molecule. For each rotatable bond in the generation trees, we collect regression targets for training the scorer by following the procedure outlined in App. A.2
Batching. When training the graph generator, we batch together graph-generative actions which are part of the same generation sequence (e.g., generating G((e) from G((C-))). Otherwise, generation sequences are treated independently. When training the rotatable bond scorer, we batch together different query dihedrals t/toe of the same focal bond. Rather than scoring all 36 rotation angles in the same batch, we include the ground-truth rotation angle and randomly sample 9/35 others to include in the batch. Within each batch (for both graph-generation and scoring), all the encoded molecules
Ms are constrained to have the same number of atoms, and all the partial molecular structures Gk( are constrained to have the same number of atoms. This restriction on batch composition is purely for convenience: the public implementation of VN-DGCNN from Deng et al. (202T) is designed to train on point clouds with the same number of points, and we construct point clouds by sampling (fixed) no points for each atom.
Training setup. We train the graph generator and the rotatable bond scorer separately.
For the graph generator, we train for 2M iterations (batches), with a maximum batch size of 400 (generation sequences). We use the Adam optimizer with default parameters. We an initial learning rate of 2.5 x 10-1, which we exponentially decay by factor of 0.9 every 50K iterations to a minimum of 5 x 10-6, We weight the auxiliary losses by Snext-shape = 10.0 and Bp 10.0. We log-linearly increase BKL from 10-5 to 10-1 over the first IM iterations, after which remains constant at 10-1 For each generation sequence, we randomize the rotation angle of the bond connecting the focus to the rest of the partial graph (e.g., the focal dihedral), as this dihedral has yet to be scored. In order to make the graph generator more robust to imperfect rotatable bond scoring at generation time, during training, we perturb the dihedrals of each rotatable bond in the partially generated structure M/ by dúb ~ N(u = 0% a = 15") while fixing the coordinates of the focus.
For the rotatable bond scorer, we train for 2M iterations (baches), with maximum batch size of 32 (focal bonds). Since we sample 10 query dihedrals per focal bond, the effective (maximum) batch size is 320. We use the Adam optimizer with default parameters. We use initial learning rate of 5 x 10-4, which we exponentially decay by a factor of 0.9 every 50K iterations to a minimum of 1 x 10-5 In order to make the scorer more robust to imperfect bond scoring earlier in generation
40

---

Published as a conference paper at ICLR 2023
sequence, during training, perturb the dihedrals of each rotatable bond in the partially generated structure Mj by 8rp NN = 0" o = 5)) while fixing the coordinates of the focus.
Training times. Training the graph generator on GPU takes 5-7 days. Training the rotatable bond scorer on GPU takes days.
Dataset processing times. Pre-computing the regression targets for the rotatable bond scorer is the most compute-intensive across 384 cpu cores.
intensive part of dataset pre- pre-processing. We required 2-3 days when parallelizing
41

---

Published as a conference paper at ICLR 2023
OH
NH2
Figure 15:
Examples of starting seed structures My highlighting where new atoms/fragments might be attached (orange).
A.16 ADDITIONAL GENERATION DETAILS
Seeding generation. When generating new molecules M' from Ms, we start by extracting a small structure Mo from Ms seed generation. In our experiments, Mo determined by the following criteria:
My has Σ heavy atoms
My contains at least terminal atom (an atom with only 1 bonded neighbor)
Any new noom/fragment that is bonded MM forms valid dihedral. For this be satisfied, Mo must contain at least atoms.
Mg does not contain sequence of atoms that forms a valid dihedral of flexible rotatable bond. Hence, all rotatable bonds scored during generation.
Figure shoos some examples of common seed structures. If there are multiple suitable Mo for given Ms. we arbitrarily select one of them. We only consider one Mo per Ms our experiments. Once we extract Mo. we fix the coordinates of Mo to their ground truth positions in Ms.
We note that the above seeding procedure could be replaced by suitable model that builds and (equivariantly) predicts the coordinates of the starting structure My. In this work, extract seeds for simplicity, but use the above criteria to ensure that the downstream shape-conditioned 3D gen- eration and design tasks non-trivial. We also emphasize that our choice of seeding procedure is fairly arbitrary, and does not necessarily restrict how the model used in other generation tasks. For instance, larger seed structures could be used.
Masking invalid actions. When generating the molecular graph, we mask actions which would violate chemical valency. Because define atom types in part by the number of single, double, and triple bonds the atom forms (including bonds to implicit hydrogens), we create valency masks by ensuring that each atom forms the correct number and types of bonds. When scoring rotatable bonds, we also mask query dihedrals that would lead to severe steric clash (<IA) with the existing partially generated molecular structure (App. A.2
Bond scoring. When scoring rotable bond at generation time, we sample 36 dihedral angles separated by 10 In principle, one could use finer discretization schemes to generate more refined 3D conformations, at the cost of speed and/or memory.
Threshold for stopping local generation. We manually tuned Ty, the threshold for local stopping, to = 0.01 in order to balance tendencies of the model to 1) prematurely stop local genera- tion (thus leading to molecules that do not completely fill the target shape), and 2) generate extra atoms/fragments around the focus (leading steric crowding).
Generation time. Typically, generating (in serial, on a cpu) one 3D molecule with 20-30 atoms takes 2-3 seconds. Overall generation time (on cpu) and memory cost significantly increase if many stereoisomers are enumerated when scoring any particular rotatable bond (App. A.2). We thus cap the number of enumerated stereoisomers to 32 (per focus).
42

---

Published as a conference paper at ICLR 2023
A.17 RELAXATION OF GENERATED GEOMETRIES
This work focuses on generating flexible drug-like molecules in specific 3D conformations that fit target shape. Notably, SQUID generates 3D conformations using heuristic bond distances and bond angles. Consequently, the generated 3D conformations are explicitly constrained to have reasonable bonding geometries (c.g., local structures). Empirically, the generated molecules also have few steric clashes, with just 13% of the generated molecules having steric clash under 2 A. Nevertheless, the generated molecules are not necessarily their minimum energy conformations. Note that this is by design, as SQUID is intentionally designed to generate molecules that fit arbitrary 3D molecular shapes. In this section, we relax the generated geometries with force field optimization and consider how this geometry relaxation affects the shape similarity to the target shape.
We use the following procedure to (locally) optimize a generated 3D conformer. Given a SQUID- generated 3D molecule, we first extract the molecule's ' 2D graph, and use RDKit to generate new (unrelated) conformation of the molecule, with explicit hydrogens included. Note that explicit hy- drogens (which are not natively generated by SQUID) are needed for accurate force-field geometry optimization. We then manually rotate each rotatable bond in the RDKit-generated conformer to the exact rotation angle that was natively generated by SQUID, thereby (approximately) yielding the same conformation that SQUID generated. The major differences between these new conformations and the original conformations are that the new conformations (1) include hydrogens, (2) have re- laxed local bonding geometries, and (3) may have slightly different fragment geometries. Note that this is similar the procedure that is performed in Appendix [A.8] except now explicit hydrogens are included in the 3D structure. We then use the MMFF force field in RDKit to optimize the new 3D conformation, using either 20 or 50 optimization steps, and finally remove the hydrogens from the optimized conformation.
Figure T6 shows the distribution of the shape similarity between original (unrelaxed) SQUID- generated (A 1.0) conformations and their geometry-optimizer counterparts. After 20 steps of MMFF optimization, approximately 68% of generated conformers have a shape similarity to their geometty-relaxed counterparts of at least 0.9. After 50 steps, this proportion only drops to 62%. Hence, the majority of generated conformations do not undergo large shape changes upon (local) geometry optimization. This indicates that the generated conformations, whilst not necessarily the minimum energy conformations, are for the most part geometrically reasonable. The largest changes in 3D shape upon geometry relaxation either occur due steric clashes in the generated conforma- tions (~~ 13% of generated molecules have a steric clash), or due to the model approximating all acyclic rotatable bonds as fully rotatable. In reality, some "rotatable" bonds actually have high bar- riers to rotation due to favorable orbital interactions. For instance, some amide bonds (such as those in peptides) prefer to be planar due to pi-orbital interactions and electron delocalization. In its cur- rent formulation, SQUID does not attempt to account for these kinds of preferred orientations of otherwise rotatable bonds.
We are especially interested in how relaxing the generated geometries affects the shape similarity to the target 3D shape. Figure 17 shows the distribution in sims(M', Mss) for the best of Nmmx = 20 SQUID- molecules M' (sampled from the prior, X 1.0) with sime (M', MM) < 0.7,
before and after the generated molecules undergo geometry optimization. Importantly. relaxing the geometries of the generated 3D molecules does not have large impact on the distribution of shape similarity to the target 3D shape. In particular, even after 50 steps of geometry optimization, the generated molecules are still significantly enriched in shape similarity to the target 3D shape compared to molecules randomly sampled from the dataset.
43

---

Published as a conference paper at ICLR 2023
20 MMFF steps
5000
50 MMFF steps
4000
Frequency
3000
2000
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Shape Similarity
Distribution of shape similarity between prior (A 1.0), and the same molecules after 20 (blue) or 50 (orange) steps of geometry relaxation with MMFF.
Figure 16:
molecules sampled from the
unrelaxed
20 MMFF steps
50 MMFF steps
Density
4
0.3
0.4
0.5
0.6
0.7
0.8
0.9
10
Shape Similarity
Figure 17: Distributions of (MM''M', MM)) for the best of Nmax = 20 SQUID-generaue (A = 1.0) molecules M' with simo(M', .Ms) 0.7, before (red) or after each M has undergone 20 (dark grey) or 50 (light grey) steps of MMFF geometry relaxation.
44

---

Published as a conference paper at ICLR 2023
A.18 COMPARISON TO LIGDREAM (SKALIC ET AL., 2019)
In this section, we compare SQUID to LigDream (Skalic et al. 2019). which is a shape-captioning network that generates ID SMILES strings of molecules conditioned on 3D CNN encoding of target 3D molecular shape and pharmacophores. In contrast to SQUID, LigDream (1) does not gen- erate 3D conformers end-to-end, and hence requires post hoc conformer generation/enumeratinn (2) is not robust to rotations, as LigDream's s 3D CNN encoder is not SO(3)-invariant; and (3) does not separately encode molecular shape and chemical identity. Nevertheless, we still compare to Lig- Dream to demonstrate SQUID's significant advantages for shape-conditioned molecular generation and design.
Skalic et al. (2019) train LigDream to convergence using > 25 million molecules from a drug-like subset of ZINCT5. Rather than re-training LigDream on our significantly smaller dataset of ~ million molecules, we directly apply Skalic et al. (2019's 's pre-trained LigDream on molecules from our test set. Note that our dataset is derived from MOSES, itself subset of ZINC15, and hence the two models are trained on very similar molecules. We find this sufficient for the purposes of this comparison, with the caveat that LigDream is still trained on 20 times more data than SQUID.
To fairly and directly compare SQUID (a 3D model) to LigDream (a 1D/2D model requiring post hoc conformer generation), we perform the following procedure. Given a reference molecule Ms with a target 3D shape, we encode Ms using both SQUID (A = 1.0) and LigDream. LigDream's encoder also employs a VAE; LigDream samples diverse molecules by controlling a "variability factor" ALieDraam- which scales the variance of LigDream's posterior distribution. Following (Skalic iet al. (019), we consider both ALigDream = 1.0 and 5.0. After encoding Ms. we use SQUID and LigDream to generate a pool of 50 molecules, which we then filter to only include those with (i'' ,Ms) < 0.7. Of the filtered molecules, we then sample Nms 20 molecules. For SQUID, we directly select the generated molecule from these Nmax molecules that maximizes sims(MM, Ms). For LigDream, we use RDKit to generate Nc. conformations for each of the Nmmx generated molecules, and select the conformer amongst these Nmax No total conformations that max- imizes sims( (M', sims(M', .M Since SQUID directly generates just conformation per generated molecule,
we use No = 1 for a head-to-hearl to-h comparison.
Using this procedure, Figure 18 plots the distribution in sims(M', Ms) for 1000 distinct Ms. using SQUID (prior, A = 1.0) and LigDream (ALigDream = 1.0, 5.0). Overall, SQUID gener- ates 3D molecules which are significantly more shape-similar to Ms versus using LigDream with AgggDream = 1.0 or 5.0. This is due to SQUID being able to directly generate 3D conformers that fit the target shape, and not relying on non-shape-conditioned conformer generator (such as RDKit).
Importantly, the SQUIID-generatdd molecules have an average chemical similarity to the encoded molecule (sima(M .Ms) of 0.26, whereas the De-DDrra-ggnerate 1.0) have a higher average chemical similarity of 0.36. This is a substantial result, as we are inter- ested in generating chemically diverse molecules that fit the target shape not generating molecules that look like Ms. Note that SQUID can always generate more chemically similar molecules by decreasing X, if desired. Increasing ALigDream to 5.0 causes LigDream to generate more chemically diverse molecules, with the average chemical similarity now on par with that obtained with SQUID. However, setting ALigDream = 5.0 causes the shape-simile to Ms. This emphasizes that 2D shape-conditions molecules to be substantially less generative models will struggle to generate chemically diverse molccules that still fit the target 3D shape, presumably because they do not simultaneously generate both the molecular graph and 3D coordinates (see Challenge 3). On the other hand, because SQUID (1) separately encodes 3D shape and 2D chemical identity, and (2) simultaneously generates both the molecular graph and the molecular coordinates, SQUID is able to generate more diverse molecules while still fitting the target 3D shape.
molecules (using ALigDraam
To test whether increasing Ne can close the gap in the shape similarity distributions between SQUID and LigDream (with ALigDream = 5.0), we increase No to 20. This essentially permits RDKit to sample more conformational space in order to search for conformations of the LigDream- generated molecules that best fit the target shape. Note that this is no longer a head-to-head compar- conformations to the bf-
ison, as we are now comparing the best-of-20 SQUID-generatte LigDream(+RDKii)-geen -generttd conformations. Nevertheless, Figure 19 shows that even when us- ing Ne = 20, SQUID still significantly outperforms LigDream in generating chemically diverse molecules with high shape similarity to the target 3D shape.
45

---

Published as a conference paper at ICLR 2023
SQUID (prior; (simg) = 0.26)
10
LigDream (A = 1.0; (simg) = 0.36)
LigDream (1 5.0: (sima) 0.27)
Density
0.3
0.4
0.5
0.6 0.7
0.8
0.9
1.0
Shape Similarity
Figure 18:
Distributions of sims MM) for the best of Nm 20
(prior,
1.0) or se-Dream-generatee (Nc = 1, 1.0 or 5.0) molecules with simg MM', Ms) < 0.7. SQUID 1.0) generates more chemically diverse molecules compared LigDream 1.0), and SQUID generates more shape similar molecules compared to LigDream (A 5.0).
SQUID (prior) LigDream () = 5.0. =0)
Density
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Shape Similarity
Figure 19: Distributions of (Mm MMs) for the best of Nmax 20 JQUID-generated (prior,
1.0) or LigDream-generated generated (Nc 20, 5.0) molecules with simg (M',Ms) Ms) 0.7.
46


Reviews:
[{"review_id": "tOOj7Lr1Z6k", "reviewer": "Reviewer_bX75", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper designs a novel framework to generate molecules regarding to the point cloud of molecules. The authors make a lot of effort into building the model consisting of VN-DGCNN, GNN, scoring rotatable bonds and so on.", "summary_of_the_review": "I tend to reject the paper because I still doubt the contribution to the machine learning community. Although I believe it shows solid contribution to the drug discovery community. This work is more suitable to a journal of computational chemistry.", "strength_and_weaknesses": "Strength:\n1. The paper proposes a complex generative system for molecular generation with very detailed task-specific model design.\n2. In order to build the molecules with 3D conformation, the proposed method introduces a novel scoring rotatable bonds to learn the rolling angles of each fragment.\n3. The proposed method resolves a critical problem as scaffold hopping in drug design.\n4. The experimental results show the benefits of the proposed method.\n\nWeakness:\n1. The contribution to the machine learning community is limited. The whole model is based on a CVAE framework with two embedding networks, VG-DGCNN and GNN and one fragment-based sequential generative decoder. The scoring rotatable bonds is a novel contribution but incremental to the ML community.\n2. The rolling bonds of the molecules should contain periodicity. However, the output of scoring rotatable bonds is the output of a sigmoid function which is not a periodic function. \n3. Since the base framework follows a CVAE, is the proposed method able to reconstruct the molecules in terms of the point cloud and 2D graph? It is better to report the validity, uniqueness, novelty, and reconstruction rate to indicate whether the model learns the information of molecules properly.\n4. I am curious given a fixed shape, what is the diversity of the generated molecules? Are all generated conformers stable at the lowest energy and still fit to the required shape?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "GebRLGZP1m", "reviewer": "Reviewer_TAEy", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In the proposed work, the authors present a method for de novo drug design conditioned on specific pharmacophores. They do so by using an autoregressive model similar to MoLeR (Maziarz et al 2021) and cleverly extending it to the 3D dimensional case with equivariant/invariant networks. ", "summary_of_the_review": "Overall, this is a good paper and believe that it is technically novel. I think, however, that it would benefit from an extended experiments section where the practical applicability of the method is compared to that of previous approaches.\n\nAdditional comments:\n* One of the main selling points of shape-conditional generation is to generate molecules similar to a specific co-crystallized structure (i.e., in structure=based drug design). The authors could potentially use structure data to generate molecules for a given pharmacophore inside a protein pocket and compare how docking scores (without redocking, only scoring) compare to those of the reference compound.\n* How is the p_{site} MLP computed? Since different fragments will have a variable number of attachment points, does that mean that each fragment type has its own separate MLP?\n* Similarly are the MLPs of p_{focus}, p_{next} and p_{site} separate? If so, do they need a subscript?\n* (On Table 5): Could the authors elaborate on why the graph reconstruction accuracy is so low? Is that a consequence of the autoregressive nature of the model? Similarly, could they comment on the amount of steric clashes (reported 13.1%)?\n* (On the shape-constrained molecular optimization section) Do the authors use an existing genetic algorithm implementation or do they implement their own?\n* Equations are lacking commas and periods when appropriate\n* Figure 2 caption has a typo. (\"encoder-decod er\")", "strength_and_weaknesses": "Strengths:\n\n* Manuscript very well written\n* First approach of its kind, where the goal is to directly generate a 3D dimensional molecule end-to-end conditioned on a specific pharmacophore.\n* Methodology shows potential for scaffold-hopping.\n* Accompanying code to facilitate method testing\n\n\nWeaknesses:\n* Evaluation limited to the proposed method itself. While the proposed method is currently unique in the sense that it's fully end-to-end, the authors could have compared to other \"multi-step\" approaches such as the ones proposed by Skalic et al. (2019) or Imrie et al. (2021), even if the latter require additional conformation generation / alignment. From a technical point of view the model is novel but its applicability advantage compared to previous approaches remains unclear.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Ad-_s6Cny7", "reviewer": "Reviewer_vKuJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper describes a method that can generate a sets of molecules that approximately match an input 3D \"shape\".  The method employs an autoregressive encoder-decoder architecture that learns a representation of the input shape (VN-DGCNN) and partial graph and decodes an updated molecule graph with an additional atom or fragment at each step (inspired by MoLeR). The method use a number of useful heuristics for bonds and angles to limit learning to the rotatable dihedral angles of the small molecule graphs. The authors demonstrate their model on a number of shape-conditioned tasks.\n", "summary_of_the_review": "This paper is a useful, original contribution that can become stronger with minor modifications.\n", "strength_and_weaknesses": "The paper is a solid contribution to a reasonably recent area of exploration, the conditional generation of small molecules with 3D coordinates.  The technical aspects of the method are well thought out and described in a clear fashion, though I list a couple of remaining questions and omissions below. The authors demonstrate their method on a reasonable number of synthetic examples, and document its level of success.\n\nOne weakness in the presentation of this paper is the lack of a clear explanation of the rationale and implications from the restriction of chemical space to the combinatorial chemical landscape defined by the chosen set of 100 fragments and their non-ring linkers. I understand the need to demonstrate a method quickly, and perhaps this was the main motivation, however, while reading this work I kept wondering if perhaps the cost of a larger search tree perhaps becomes prohibitive and renders the method impractical. If that's not the case, would it be possible to simply generate molecules atom-by-atom, similar to previous 3D generation models, and let the model learn to enter rings and keep generating ring atoms until they close and so on? Alternatively, could one increase the size of fragments dramatically to include all possible single, double, and triple rings in any of the currently available patents?  Would such a change degrade the reconstruction quality?  Approximately how would the time for generation of a molecule (currently listed as 2--3 seconds of walltime) scale if one increased the size of the fragment library by a factor of 2, 5, 10, 100?\n\nIn terms of coordinates, the model is only learning to generate the rotatable dihedrals, which sounds like a rational way to evaluate a shape-matching model.  However, I wonder how much strain these dihedrals undergo in the final proposed conformation.  Instead of using the procedure to generate Figure 12, could the authors find the local minimum of the MMFF (the force field they used for generating the input conformations) and show the statistics of the shape similarity for these relaxed configurations?  \n\nCould the model drop the dependence on the initial fragment by learning the distribution of such initial fragments in the training set and learning to condition their initial placements on the shape?  I'd imagine that this problem is much easier than learning how to construct a reasonable search tree, and if it is harder than I imagine, then perhaps the authors could use an existing conditional pose generator, deep (or shallow) docking tool, or something else that is efficient as a starting point.  In that way, there will be no need to disclose any information about the test molecule that generated the shape for the final shape-matching task. \n\nA confusing mistake in the paper is the improper description of the work in the pocket2mol paper, which also conditions on 3D inputs (the pocket) and generates complete molecules (not only fragments).  I strongly recommend that the authors improve their otherwise clear review of the related literature at the time of their submission, and amend the relevant clause in the second sentence of the abstract.\n\nFinally, a question that doesn't need to be addressed in the paper, however, I would appreciate any thoughts that the authors might have on the subject below, and perhaps their answer could improve the paper after all.  The method that the authors use to align the molecules, ROCS, also has a GPU implementation that is reasonably scalable and efficient, and could thus render other, less specialized algorithms competitive (e.g. a traditional genetic algorithm that mixes graphs to iteratively optimize the shape scores in an ensemble, or a naive iterative trainable 3D molecule generator that scores and filters partial molecule graphs.)  Additionally, ROCS also can align and score molecules using both shape and color (chemical features)---I expect that a generalization that addresses color in addition to shape would be relevant in a practical drug discovery setting.  It would be trivial to modify an ensemble optimization algorithm to score via shape+color ROCS, however, typical ML methods would require training from scratch.  Is there a missing flexibility in the current wave of neural-network-based acceleration of chemoinformatics tasks and are there any generic ways to go around them to help build flexible building blocks (similar to what VNs did for building SO3 equivariant networks, but now for small molecule drug discovery)?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "9MQk2Mmdcp", "reviewer": "Reviewer_UfVq", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studied shape-conditioned 3D molecule generation, which aims to generate molecules with a desirable shape. The author proposed an encoder-decoder architecture, where the encoder can encode both molecular graph representation and molecular shape representation by point clouds. Specifically, the main difference between the proposed model and existing molecular generative models is additionally taking shape (point clouds) as model inputs. Experiments show that the model can enable the shape-constrained generation and optimization of molecules.", "summary_of_the_review": "The paper solves a new task of shape-conditioned molecule generation. I suggest significantly refining the content to make the paper easier to follow. Besides, comparing recent state-of-the-art VAE-based molecular generative models to justify the empirical improvements (see weakness 3).", "strength_and_weaknesses": "Strength:\n\n1. The tackled problem is new for the \"ML for molecule\" community, which hasn't been explored in previous literature.\n2. The chemical background and challenges of this task are greatly explained, making the content friendly to the general ML audience.\n3. The authors conduct comprehensive experiments and ablations to justify many different \n\nWeakness:\n\n1. Overall, the paper is a little hard to follow. As in Fig.2 and Sec.3, the author introduced too many intermediate variables within the neural network parameterizations, with just minor differences in subscripts. This is not very informative and makes the content not coherent enough. I suggest for the methodology part, try to just simply summarize the parameterization of the individual encoders & decoders; besides, make key parts such as the objective function and generation tree more clear and self-contained in the paper.\n2. As shown in Sec.3 problem definition paper, the author claims to generate molecules with \"similar shape\" and \"low similarity molecular representation\". However, I think overall the proposed model just learns to generate existing molecules, without new objectives or RL  methods to explicitly encourage the exploration. The major difference is taking shape as additional inputs. However, as shown in Appendix A.9, actually the performance improvement by taking shapes as inputs are not significant.\n3. Furthermore, the model actually is a combination of \"molecular graph generative models\" and \"torsion scoring models\". This makes me feel like you should also take existing molecular graph generation VAE models as baselines, such as [1]. For example, input $M_S$ into JT-VAE's encoder, and test whether decoder-generated molecules can maybe have competitive results with your model.\n\n[1] Jin, Wengong, Regina Barzilay, and Tommi Jaakkola. \"Junction tree variational autoencoder for molecular graph generation.\" In International conference on machine learning, pp. 2323-2332. PMLR, 2018.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster