Learning 3D Representations of Molecular Chirality with Invariance to Bond Rotations

1 INTRODUCTION . Advances in graph neural networks ( GNNs ) have revolutionized molecular representation learning for ( bio ) chemical applications such as high-fidelity property prediction ( Huang et al. , 2021 ; Chuang et al. , 2020 ) , accelerated conformer generation ( Ganea et al. , 2021 ; Mansimov et al. , 2019 ; Simm & Hernandez-Lobato , 2020 ; Xu et al. , 2021 ; Pattanaik et al. , 2020b ) , and molecular optimization ( Elton et al. , 2019 ; Brown et al. , 2019 ) . Fueling recent developments have been efforts to model shape-dependent physio-chemical properties by learning directly from molecular conformers ( snapshots of 3D molecular structures ) or from 4D conformer ensembles , which better capture molecular flexibility . For instance , recent state-of-the-art ( SOTA ) GNNs feature message updates informed by bond distances , bond angles , and torsion angles of the conformer ( Schütt et al. , 2017 ; Klicpera et al. , 2020b ; Liu et al. , 2021 ; Klicpera et al. , 2021 ) . However , few studies have considered the expressivity of GNNs when tasked with learning the nuanced effects of stereochemistry , which describes how the relative arrangement of atoms in space differ for molecules with equivalent graph connectivity . Tetrahedral ( point ) chirality is among the most prevalent types of stereochemistry , and describes the spatial arrangement of chemical substituents around the vertices of a tetrahedron centered on a chiral center , typically a carbon atom with four non-equivalent bonded neighbors . Two molecules which differ only in the relative atomic arrangements around their chiral centers are called stereoisomers , or enantiomers if they can be interconverted through reflection across a plane . Enantiomers are distinguished in chemical line drawings by a dashed or bold wedge indicating whether a bonded neighbor to a chiral center is directed into or out of the page ( Figure 1A ) . Although enantiomers share many chemical properties such as boiling/melting points , electronic energies ( e.g. , from QM9 ( Ramakrishnan et al. , 2014 ) ) , and solubility in most solvents , enantiomers can display strikingly different behavior when interacting with external chiral environments . For instance , chirality is critical for pharmaceutical drug design ( Nguyen et al. , 2006 ; Jamali et al. , 1989 ) , where protein-ligand interactions may be highly influenced by ligand chirality , as well for designing structure-directing agents for zeolite growth ( Luis & Beatriz , 2018 ) and for optimizing enantioselective catalysts ( Pfaltz & Drury , 2004 ; Liao et al. , 2018 ) . Chiral centers are inverted upon reflection through a mirror plane . Consequently , E ( 3 ) -invariant 3D GNNs that only consider pairwise atomic distances or bond angles in their message updates , such as SchNet ( Schütt et al. , 2017 ) and DimeNet/DimeNet++ ( Klicpera et al. , 2020a ; b ) , are inherently limited in their ability to distinguish enantiomers ( Figure 1B ) . Although SE ( 3 ) -invariant 3D GNNs , such as the recently proposed SphereNet ( Liu et al. , 2021 ) and GemNet ( Klicpera et al. , 2021 ) , can in theory learn chirality , their expressivity in this setting has not been explored . Alongside the development of 3D GNNs , which process individual 3D conformers , there have been efforts to better represent conformational flexibility by encoding multiple conformers in a 4D ( multi-instance ) ensemble for property prediction ( Zankov et al. , 2021 ; 2019 ; Axelrod & GomezBombarelli , 2021 ) , identifying important conformer poses ( Chuang & Keiser , 2020 ) , and predicting solvation energies ( Weinreich et al. , 2021 ) . Unless in the solid state , molecules are not rigid objects or static 2D graphs , but are flexible structures that continuously interconvert through rapid rotations of chemical bonds as well as through a number of smaller perturbations such as bond stretching , bending , and wagging . Explicitly modeling this probability distribution over accessible conformer space has the potential to drastically improve the modeling of protein-drug interactions , where the most relevant active pose of the ligand is not known a priori , as well as in the prediction of Boltzmann-averages , which depend on a distribution of conformers . One challenge with these methods is selecting which conformers to include in an ensemble : the space of accessible conformations combinatorially explodes with the number of rotatable bonds , and important poses are not known a priori . Modeling flexibility with multi-instance methods thus requires explicit conformer enumeration , increasing the cost of training/inference without guaranteeing performance gains . Apart from 2D methods , which ignore 3D information altogether , no studies have explicitly modeled conformational flexibility directly within a model architecture . To explicitly model tetrahedral chirality and conformational flexibility , we design a neural framework to augment 2D GNNs with processing of the SE ( 3 ) -invariant internal coordinates of a conformer , namely bond distances , bond angles , and torsion angles . Our specific contributions are : • We design a method for graph neural networks to learn the relative orientations of substituents around tetrahedral chiral centers directly from 3D torsion angles • We introduce a novel invariance to internal rotations of rotatable bonds directly into a model architecture , potentially mitigating the need for 4D ensemble methods or conformer-based data augmentation to treat conformational flexibility of molecules • We propose a contrastive learning framework to probe the ability of SE ( 3 ) -invariant 3D graph neural networks to differentiate stereoisomers in a learned latent space • Through our ablation study , we demonstrate that a global node aggregation scheme , adapted from Winter et al . ( 2021 ) , which exploits subgraphs based on internal coordinate connectivity can provide a simple way to improve GNNs for chiral property prediction We explore multiple tasks to benchmark the ability of our model to learn the effects of chirality . We do not consider common MoleculeNet ( Wu et al. , 2017 ) benchmarks , as our focus is on tasks where the effects of chirality are more distinguishable from experimental noise . Our self-supervised contrastive learning task is the first of its kind applied to clustering multiple 3D conformers of different stereoisomers in a latent space . Following Pattanaik et al . ( 2020a ) , we also employ a toy R/S labeling task as a necessary but not sufficient test of chiral recognition . For a harder classification task , we follow Mamede et al . ( 2021 ) in predicting how enantiomers experimentally rotate circularly polarized light . Lastly , we create a dataset of simulated docking scores to rank small enantiomeric ligands by their binding affinities in a chirality-sensitive protein pocket . We make our datasets for the contrastive learning , R/S classification , and docking tasks available to the public . Comparisons with 2D baselines and the SE ( 3 ) -invariant SphereNet demonstrate that our model , Chiral InterRotoInvariant Neural Network ( ChIRo ) , achieves SOTA in 3D chiral molecular representation learning . 2 RELATED WORK . Message passing neural networks . Gilmer et al . ( 2017 ) introduced a framework for using GNNs to embed molecules into a continuous latent space for property prediction . In the typical 2D message passing scheme , a molecule is modeled as a discrete 2D graph G with atoms as nodes and bonds as edges . Nodes and edges are initialized with features xi and eij to embed initial node states : h0i = U0 ( xi , { eij } j∈N ( i ) ) ( 1 ) where N ( i ) denotes the neighboring atoms of node i . In each layer t of message passing , node states are updated with aggregated messages from neighboring nodes . After T layers , graph feature vectors are constructed from some ( potentially learnable ) aggregation over the learned node states . A readout phase then uses this graph embedding for downstream property prediction : ht+1i = Ut ( h t i , m t+1 i ) , m t+1 i = ∑ j∈N ( i ) Mt ( h t i , h t j , eij ) ( 2 ) ŷ = Readout ( g ) , g = Agg ( { hTi |i ∈ G } ) ( 3 ) There exist many variations on this basic message passing framework ( Duvenaud et al. , 2015 ; Kearnes et al. , 2016 ) . In particular , Yang et al . ( 2019 ) ’ s directed message passing neural network ( DMPNN , or ChemProp ) based on Dai et al . ( 2016 ) learns edge-based messages mtij and updates edge embeddings htij rather than node embeddings . The Graph Attention Network ( Veličković et al. , 2018 ) constructs message updates mti using attention pooling over local node states . 3D Message Passing and Euclidean Invariances . 3D GNNs differ in their message passing schemes by using molecular internal coordinates ( distances , angles , torsions ) to pass geometryinformed messages between nodes . It is important to use 3D information that is at least SE ( 3 ) invariant , as molecular properties are invariant to global rotations or translations of the conformer . SchNet ( Schütt et al. , 2017 ) , a well-established network for learning quantum mechanical properties of 3D conformers , updates node states using messages informed by radial basis function expansions of interatomic distances between neighboring nodes . DimeNet ( Klicpera et al. , 2020b ) and its newer variant DimeNet++ ( Klicpera et al. , 2020a ) exploit additional molecular geometry by using spherical Bessel functions to embed angles ϕijk between the edges formed between nodes i and j and nodes j and k ̸= i in the directed message updates to node i. SchNet and DimeNet are E ( 3 ) -invariant , as pairwise distances and angles formed between two edges are unchanged upon global rotations , translations , and reflections . Since enantiomers are mirror images , SchNet and DimeNet are therefore invariant to this form of chirality . To be SE ( 3 ) -invariant , 3D GNNs must consider torsion angles , denoted ψixyj , between the planes defined by angles ϕixy and ϕxyj , where i , x , y , j are four sequential nodes along a simple path . Torsion angles are negated upon reflection , and thus models considering all torsion angles should be implicitly sensitive to chirality . Flam-Shepherd et al . ( 2020 ) , Liu et al . ( 2021 ) ( SphereNet ) , and Klicpera et al . ( 2021 ) ( GemNet ) introduce 3D GNNs that all embed torsions in their message updates . Using a complete set of torsion angles provides access to the full geometric information present in the conformer but does not guarantee expressivity when learning chiral-dependent functions . Torsions are negated upon reflection , but any given torsion can also be changed via simple rotations of a rotatable bond–which changes the conformation , but not the molecular identity ( i.e. , chirality does not change ) . Reflecting a non-chiral conformer will also negate its torsions , but the reflected conformer can be reverted to its original structure via rotations about internal bonds . To understand chirality , neural models must learn how coupled torsions , the set of torsions { ψixyj } ( i , j ) that share a bond between nodes x and y ( with x or y being chiral ) , collectively differ between enantiomers . E ( 3 ) - and SE ( 3 ) -Equivariant Neural Networks . Recent work has introduced equivariant layers into graph neural network architectures to explicitly model how global rotations , translations , and ( in some cases ) reflections of a 3D structure transform tensor properties , such as molecular dipoles or force vectors . SE ( 3 ) -equivariant models ( Fuchs et al. , 2020 ; Thomas et al. , 2018 ) should be sensitive to chirality , while E ( 3 ) -equivariant models ( Satorras et al. , 2021 ) will only be sensitive if the output layer is not E ( 3 ) -invariant . Since we use SE ( 3 ) -invariant internal coordinates as our 3D representation , we only compare our model to other SE ( 3 ) - or E ( 3 ) -invariant 3D GNNs . Explicit representations of chirality in machine learning models . A number of machine learning studies account for chirality through hand-crafted molecular descriptors ( Schneider et al. , 2018 ; Golbraikh et al. , 2001 ; Kovatcheva et al. , 2007 ; Valdés-Martinı́ et al. , 2017 ; Mamede et al. , 2021 ) . A naı̈ve but common method for making 2D GNNs sensitive to chirality is through the inclusion of chiral tags as node features . Local chiral tags describe the orientation of substituents around chiral centers ( CW or CCW ) given an ordered list of neighbors . Global chiral tags use the Cahn-IngoldPrelog ( CIP ) rules for labeling the handedness of chiral centers as R ( “ rectus ” ) or S ( “ sinister ” ) . It is unclear whether ( and how ) models can suitably learn chiral-dependent functions when exposed to these tags as the only indication of chirality . Pattanaik et al . ( 2020a ) propose changing the symmetric message aggregation function in 2D GNNs ( sum/max/mean ) to an asymmetric function tailored to tetrahedral chiral centers , but this method does not learn chirality from 3D molecular geometries . Chirality in 2D Vision and 3D Pose Estimation . Outside of molecular chirality , there has been work in the deep learning community to develop neural methods that learn chiral representations for 2D image recognition ( Lin et al. , 2020 ) and 3D human pose estimation ( Yeh et al. , 2019 ) . In particular , Yeh et al . ( 2019 ) consider integrating equivariance to chiral transforms directly into neural architectures including feed forward layers , LSTMs/GRUs , and convolutional layers .

Summary: The paper focuses on improving the capacity of GNN models, using chirality identification as the case study. As an important character to represent the geometry in molecules, chirality has a fundamental impact on the molecule properties and downstream applications. It is a major extension of [1]. The distinguishing features are mainly based on the sinusoid transformations of torsion angles, which are invariant to the bond rotations.