GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles

1 Overview Problem & importance . We tackle the problem of molecular conformer generation ( MCG ) , i.e. , predicting the ensemble of low-energy 3D conformations of a small molecule solely based on the molecular graph ( fig . 1 ) . A single conformation is represented by the list of 3D coordinates for each atom in the respective molecule . In this work , we assume that the low-energy states are implicitly defined by the given dataset , i.e. , our training data consist of molecular graphs and corresponding sets of energetically favorable 3D conformations . Low-energy structures are the most stable configurations and , thus , expected to be observed most often experimentally . ⇤Equal first authorship and contribution . Correspondence to { oct , lagnajit } @ mit.edu . †Department of Chemical Engineering , MIT , Cambridge , MA 02139 ‡Computer Science and Artificial Intelligence Laboratory , MIT , Cambridge , MA 02139 4Code is available at https : //github.com/PattanaikL/GeoMol . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Dealing with molecules in their natural 3D structure is of great importance in areas such as cheminformatics or computational drug discovery because conformations determine biological , chemical , and physical properties [ Guimaraes et al. , 2012 , Schütt et al. , 2018 , Klicpera et al. , 2019 , Axelrod and Gomez-Bombarelli , 2020b , Schütt et al. , 2021 , Liu et al. , 2021 ] such as charge distribution , potential energy , docking poses [ McGann , 2011 ] , shape similarity [ Kumar and Zhang , 2018 ] , pharmacophore searching [ Schwab , 2010 ] , or descriptors for 3/4D QSAR [ Verma et al. , 2010 ] . For instance , in drug design it is crucial to understand how a molecule binds to a specific target protein ; this process heavily depends on the 3D structures of the two components , both in terms of geometric ( shape matching ) and chemical ( hydrophobic/hydrophilic ) interactions [ Gainza et al. , 2020 , Sverrisson et al. , 2020 ] . Motivation & challenges of existing methods . The main challenge in MCG comes from the enormous size of the 3D structure space consisting of bond lengths , bond angles , and torsion angles . It is known that the molecular graph imposes specific constraints on possible 3D conformations , e.g. , bond length ranges depend on the respective bond types , while tetrahedral centers dictate local spatial arrangement . However , the space of possible conformations grows exponentially with the graph size and number of rotatable bonds , thus hindering exhaustive brute force exploration even for relatively small molecules . Additionally , the number of plausibly-stable low-energy states is unknown a priori and can vary between one and several thousand conformations for a single molecule [ Chan et al. , 2021 ] . Nevertheless , various facets of the curse of dimensionality have been favorably tackled by ML models in different contexts , and our goal is to build on the recent ML efforts for MCG [ Mansimov et al. , 2019 , Simm and Hernandez-Lobato , 2020 , Lemm et al. , 2021 , Xu et al. , 2021 ] . Molecular conformations can be determined experimentally , but existing techniques are very expensive . As a consequence , predictive computational models have been developed over the past few decades , traditionally being categorized as either stochastic or systematic ( rule-based ) methods [ Hawkins , 2017 ] . Stochastic approaches have traditionally been based on molecular dynamics ( MD ) or Markov chain Monte Carlo ( MCMC ) techniques , potentially combined with genetic algorithms ( GAs ) . They can do extensive explorations of the energy landscape and accurately sample equilibrium structures , but quickly become prohibitively slow for larger molecules [ Shim and MacKerell Jr , 2011 , Ballard et al. , 2015 , De Vivo et al. , 2016 , Hawkins , 2017 ] , e.g. , they require several CPU minutes for a single drug-like molecule . Moreover , stochastic methods have difficulties sampling diverse and representative conformers , prioritizing quantity over quality . On the other hand , rule-based systematic methods achieve state-of-the-art in commercial software [ Friedrich et al. , 2017 ] with OMEGA [ Hawkins et al. , 2010 , Hawkins and Nicholls , 2012 ] being a popular example . They usually process a single drug-like molecule under a second . They address the aforementioned challenges of stochastic methods by relying on carefully curated torsion templates ( torsion rules ) , rule-based generators , and knowledge bases of rigid 3D fragments , which are assembled together and combined with subsequent stability score ranking . However , torsion angles are mostly varied independently ( based on their fragments ) , without explicitly capturing their global interactions , which results in difficulties for larger and more flexible molecules . Furthermore , the curated fragments and rules are inadequate for more challenging inputs ( e.g. , transition states or open-shell molecules ) . Both types of methods can be combined with Distance Geometry ( DG ) techniques to generate the initial 3D conformation . First , the 3D atom distance matrix is generated based on a set of distance constraints or from a specialized model . Subsequently , the corresponding 3D atom coordinates are learned to approximately match these predicted distances [ Havel et al. , 1983b , a , Crippen et al. , 1988 , Havel , 1998 , Lagorce et al. , 2009 , Riniker and Landrum , 2015 ] . Indeed , modern stochastic algorithms are entirely based on DG methods [ Riniker and Landrum , 2015 ] . The inductive bias of rotational and translational invariance is guaranteed for DG , thus being appealing for ML models [ Simm and Hernandez-Lobato , 2020 , Xu et al. , 2021 , Pattanaik et al. , 2020b ] . However , several drawbacks weaken this important direction : i ) the distance matrix is overparameterized compared to the actual number of degrees of freedom , ii ) it is difficult to enforce 3D Euclidean distance constraints as well as geometric graph constraints ( e.g. , on torsion angles or rings [ Riniker and Landrum , 2015 ] ) ; iii ) important aspects of molecular geometry are not explicitly modeled , e.g. , torsion angles of rotatable bonds or tetrahedral centers ; iv ) expensive force-field energy fine-tuning of the generated conformers is vital for a reasonable quality [ Xu et al. , 2021 , Simm and Hernandez-Lobato , 2020 ] ; iv ) the resulting multi-stage pipeline is prone to error accumulation as opposed to an end-to-end model . Previous methods often rely on a force field ( FF ) energy function minimization to fine-tune the conformers . These are hand-designed energy models which use parameters estimated from experi- ment and/or computed from quantum mechanics ( e.g. , Universal Force Field [ Rappé et al. , 1992 ] , Merck Molecular Force Field [ Halgren , 1996 ] ) . However , FFs are crude approximations of the true molecular potential energy surface [ Kanal et al. , 2018 ] , limited in the interactions they can capture in biomolecules due to their strong assumptions [ Barman et al. , 2015 ] . In addition , FF energy optimization is relatively slow and increases error accumulation in a multi-pipeline method . Relation to protein folding . There has been impressive recent progress on modeling protein folding dynamics [ Ingraham et al. , 2018 , AlQuraishi , 2019 , Noé et al. , 2019 , Senior et al. , 2020 ] , where crystallized 3D structures are predicted solely from the amino-acid sequence using ML methods . However , molecules pose unique challenges , being highly branched graphs containing cycles , different types of bonds , and chirality information . This makes protein folding approaches not readily transferable to general molecular data . Our key contributions & model in a nutshell . In this work , we investigate the question : Can we design a fast and generalizable deep learning model to predict high-quality , representative , and diverse 3D conformational ensembles from input molecular graphs ? To tackle this question , we propose GEOMOL ( shown in fig . 2 ) , exhibiting the following merits : • It is end-to-end trainable , non-autoregressive , and does not rely on DG techniques ( thus avoiding aforementioned drawbacks ) . More precisely , it outputs a minimal set of geometric quantities ( i.e. , angles and distances ) sufficient for full deterministic reconstruction of the 3D conformer . • It models conformers in an SE ( 3 ) -invariant ( translation/rotation ) manner by design . This desirable inductive bias was previously either achieved using multi-step DG methods [ Simm and HernandezLobato , 2020 ] or not captured at all [ Mansimov et al. , 2019 ] . • It explicitly models and predicts essential molecular geometry elements : torsion angles and local 3D structures ( bond distances and bond angles adjacent to each atom ) . Together with the input molecular graph , these are used for k-hop distance computation at train time and full deterministic conformation assembly at test time . Crucially , we do not over-parameterize these predictions , i.e. , a single torsion angle is computed per each non-terminal bond , irrespective of the number and permutation of the neighboring atoms at each end-point of the respective bond . • The above geometric elements ( torsion angles , local structures ) are SE ( 3 ) -invariant ( by definition or usage ) and we jointly predict them using MPNNs [ Gilmer et al. , 2017 ] and self-attention networks . Thus , unlike [ Mansimov et al. , 2019 ] , we are not affected by MPNNs ’ pitfalls that obstruct direct predictions of 3D atom coordinates from node embeddings , e.g. , symmetric or locally isomorphic nodes would always have identical MPNN embeddings [ Xu et al. , 2019 , Garg et al. , 2020 ] and , as a consequence , would be inappropriately assigned identical 3D coordinates . • To promote diverse conformer ensembles with good coverage , we devise a tailored generative loss that does not use slow or difficult-to-optimize adversarial training techniques . Using optimal transport , GEOMOL finds the best matching between generated and ground truth conformers based on their pairwise log-likelihood loss , requiring only minimization . • It explicitly and deterministically distinguishes reflected structures ( enantiomers ) by solving tetrahedral stereocenters using oriented volumes and local chiral descriptors , bypassing the need for iterative optimization usually done in DG approaches . • Empirically , we conduct experiments on two benchmarks : GEOM-QM9 ( smaller molecules relevant to gas-phase chemistry ) and GEOM-DRUGS ( drug-like molecules ) [ Axelrod and GomezBombarelli , 2020a ] . Our method often outperforms previous ML and two popular open-source or commercial methods in different metrics . Moreover , we show competitive quality even without the frequently-used computationally-demanding fine-tuning FF strategies . • GEOMOL processes drug-like molecules in seconds or less , being orders of magnitude faster than popular baselines ( e.g. , ETKDG/RDKit [ Riniker and Landrum , 2015 ] ) , without sacrificing quality . 2 Method . Problem setup & notations . Our input is any molecular graph G = ( V , E ) with node and edge features , xv 2 Rf , 8v 2 V and eu , v 2 Rf 0 , 8 ( u , v ) 2 E representing atom types , formal charges , bond types , etc . For each molecular graph G , we have a variable-size set of low-energy ground truth 3D conformers { C⇤l } l that we predict with a model { Ck } k def = ⇣ ( G ) . A conformer is a map C : V ! R3 from graph nodes to 3D coordinates , but a simplified notation is cv 2 R3 for v 2 V . We use additional notations : d ( X , Y ) = kcX cY k is the 3D distance between X and Y ; \XY Z is the counter-clockwise ( CCW ) angle \cXcY cZ ; \ ( XY Z , XY T ) is the CCW dihedral angle of the 2D planes cXcY cZ and cXcY cT ( formula is in appendix C ) . We use the corresponding c ⇤ v , d ⇤ ( X , Y ) , \⇤XY Z , \⇤ ( XY Z , XY T ) when manipulating a ground truth conformer . Any conformer is defined up to a SE ( 3 ) transformation , i.e. , any translation or rotation applied to the set { cv } v2V . A classic conformer distance function that satisfies this constraint is root-mean-square deviation of atomic positions ( RMSD ) , computed by the Kabsch alignment algorithm [ Kabsch , 1976 ] .

Summary: This paper introduces GeoMol, an end-to-end learning method for molecular conformation generation.  It is based on the construction and alignment of elements of local atomic structure, is efficient in its parameterization of geometric degrees of freedom, and is SE(3) invariant.  It achieves competitive performance on the GEOM-DRUGS and GEOM-QM9 tasks.