Geometric Transformers for Protein Interface Contact Prediction

1 INTRODUCTION . Interactions of proteins often reflect and directly influence their functions in molecular processes , so understanding the relationship between protein interaction and protein function is of utmost importance to biologists and other life scientists . Here , we study the residue-residue interaction between two protein structures that bind together to form a binary protein complex ( i.e. , dimer ) , to better understand how these coupled proteins will function in vivo . Predicting where two proteins will interface in silico has become an appealing method for measuring the interactions between proteins since a computational approach saves time , energy , and resources compared to traditional methods for experimentally measuring such interfaces ( Wells & McClendon ( 2007 ) ) . A key motivation for determining these interface contacts is to decrease the time required to discover new drugs and to advance the study of newly designed proteins ( Murakami et al . ( 2017 ) ) . Existing approaches to interface contact prediction include classical machine learning and deep learning-based methods . These methods traditionally use hand-crafted features to predict which inter-chain pairs of amino acid residues will interact with one another upon the binding of the two protein chains , treating each of their residue pairs as being independent of one another . Recent work on interface prediction ( Liu et al . ( 2020 ) ) , however , considers the biological insight that the interaction between two inter-chain residue pairs depends not only on the pairs ’ features themselves but also on other residue pairs ordinally nearby in terms of the protein complex ’ s sequence . As such , the problem of interface contact prediction became framed as one akin to image segmentation 1Training and inference code as well as pre-trained models are available at https : //github.com/BioinfoMachineLearning/DeepInteract or object detection , opening the door to innovations in interface contact prediction by incorporating the latest techniques from computer vision . Nonetheless , up to now , no works on partner-specific protein interface contact prediction have leveraged two recent innovations to better capture geometric shapes of protein structures and long-range interactions between amino acids important for accurate prediction of protein-protein interface contacts : ( 1 ) geometric deep learning for evolving proteins ’ geometric representations and ( 2 ) graphbased self-attention similar to that of Vaswani et al . ( 2017 ) . Towards this end , we introduce DeepInteract , an end-to-end deep learning pipeline for protein interface prediction . DeepInteract houses the Geometric Transformer , a new graph transformer designed to exploit protein structure-specific geometric properties , as well as a dilated convolution-based interaction module adapted from Chen et al . ( 2021 ) to predict which inter-chain residue pairs comprise the interface between the two protein chains . In response to the exponential rate of progress being made in predicting protein structures in silico , we trained DeepInteract end-to-end using DIPS-Plus ( Morehead et al . ( 2021 ) ) , to date the largest feature-rich dataset of protein complex structures for machine learning of protein interfaces , to close the gap on a proper solution to this fundamental problem in structural biology . 2 RELATED WORK . Over the past several years , geometric deep learning has become an effective means of automatically learning useful feature representations from structured data ( Bronstein et al . ( 2021 ) ) . Previously , geometric learning algorithms like convolutional neural networks ( CNNs ) and graph neural networks ( GNNs ) have been used to model molecules and to predict protein interface contacts . Schütt et al . ( 2017 ) introduced a deep tensor neural network designed for molecular tasks in quantum chemistry . Fout et al . ( 2017 ) designed a siamese GNN architecture to learn weight-tied feature representations of residue pairs . This approach , in essence , processes subgraphs for each residue in each complex and aggregates node-level features locally using a nearest-neighbors approach . Since this partner-specific method derives its training dataset from Docking Benchmark 5 ( DB5 ) ( Vreven et al . ( 2015 ) ) , it is ultimately data-limited . Townshend et al . ( 2019 ) represent interacting protein complexes by voxelizing each residue into a 3D grid and encoding in each grid entry the presence and type of the residue ’ s underlying atoms . This partner-specific encoding scheme captures static geometric features of interacting complexes , but it is not able to scale well due to its requiring a computationally-expensive spatial resolution of the residue voxels to achieve good results . Continuing the trend of applying geometric learning to protein structures , Gainza et al . ( 2020 ) developed MaSIF to perform partner-independent interface region prediction . Likewise , Dai & Bailey-Kellogg ( 2021 ) do so with an attention-based GNN . These methods learn to perform binary classification of the residues in both complex structures to identify regions where residues from both complexes are likely to interact with one another . However , because these approaches predict partner-independent interface regions , they are less likely to be useful in helping solve related tasks such as drug-protein interaction prediction and protein-protein docking ( Ahmad & Mizuguchi ( 2011 ) ) . Liu et al . ( 2021a ) created a graph neural network for predicting the effects of mutations on protein-protein binding affinities , and , more recently , Costa et al . ( 2021 ) introduced a Euclidean equivariant transformer for protein docking . Both of these methods may benefit from the availability of precise interface predictors by using them to generate contact maps as input features . To date , one of the best result sets obtained by any model for protein interface contact prediction comes from Liu et al . ( 2020 ) where high-order ( i.e . sequential and coevolution-based ) interactions between residues are learned and preserved throughout the network in addition to static geometric features initially embedded in the protein complexes . However , this work , like many of those preceding it , undesirably maintains the trend of reporting model performance in terms of the median area under the receiver operating characteristic which is not robust to extreme class imbalances as often occur in interface contact prediction . In addition , this approach is data-limited as it uses the DB5 dataset and its predecessors to derive both its training data and makes use of only each residue ’ s carbon-alpha ( Cα ) atom in deriving its geometric features , ignoring important geometric details provided by an all-atom view of protein structures . Our work builds on top of prior works by making the following contributions : • We provide the first example of graph self-attention applied to protein interface contact prediction , showcasing its effective use in learning representations of protein geometries to be exploited in downstream tasks . • We propose the new Geometric Transformer which can be used for tasks on 3D protein structures and similar biomolecules . For the problem of interface contact prediction , we train the Geometric Transformer to evolve a geometric representation of protein structures simultaneously with protein sequence and coevolutionary features for the prediction of inter-chain residue-residue contacts . In doing so , we also demonstrate the merit of the recently-released Enhanced Database of Interacting Protein Structures ( DIPS-Plus ) for interface prediction ( Morehead et al . ( 2021 ) ) . • Our experiments on challenging protein complex targets demonstrate that our proposed method , DeepInteract , achieves state-of-the-art results for interface contact prediction . 3 DATASETS . The current opinion in the bioinformatics community is that protein sequence features still carry important higher-order information concerning residue-residue interactions ( Liu et al . ( 2020 ) ) . In particular , the residue-residue coevolution and residue conservation information obtained through multiple sequence alignments ( MSAs ) has been shown to contain powerful information concerning intra-chain and even inter-chain residue-residue interactions as they yield a compact representation of residues ’ coevolutionary relationships ( Jumper et al . ( 2021 ) ) . Keeping this in mind , for our training and validation datasets , we chose to use DIPS-Plus ( Morehead et al . ( 2021 ) ) , to our knowledge the largest feature-rich dataset of protein complexes for protein interface contact prediction to date . In total , DIPS-Plus contains 42,112 binary protein complexes with positive labels ( i.e. , 1 ) for each inter-chain residue pair that are found within 6 Å of each other in the complex ’ s bound ( i.e. , structurally-conformed ) state . The dataset contains a variety of rich residuelevel features : ( 1 ) an 8-state one-hot encoding of the secondary structure in which the residue is found ; ( 2 ) a scalar solvent accessibility ; ( 3 ) a scalar residue depth ; ( 4 ) a 1 × 6 vector detailing each residue ’ s protrusion concerning its side chain ; ( 5 ) a 1 × 42 vector describing the composition of amino acids towards and away from each residue ’ s side chain ; ( 6 ) each residue ’ s coordinate number conveying how many residues to which the residue meets a significance threshold , ( 7 ) a 1 × 27 vector giving residues ’ emission and transition probabilities derived from HH-suite3 ( Steinegger et al . ( 2019 ) ) profile hidden Markov models constructed using MSAs ; and ( 8 ) amide plane normal vectors for downstream calculation of the angle between each intra-chain residue pair ’ s amide planes . To compare the performance of DeepInteract with that of state-of-the-art methods , we select 32 homodimers and heterodimers from the test partition of DIPS-Plus to assess each method ’ s competency in predicting interface contacts . We also evaluate each method on 14 homodimers and 5 heterodimers with PDB structures publicly available from the 13th and 14th sessions of CASP-CAPRI ( Lensink et al . ( 2019 ) , Lensink et al . ( 2021 ) ) as these targets are considered by the bioinformatics community to be challenging for existing interface predictors . For any CASP-CAPRI test complexes derived from multimers ( i.e. , protein complexes that can contain more than two chains ) , to represent the complex we chose the pair of chains with the largest number of interface contacts . Finally , we use the traditional 55 test complexes from the DB5 dataset ( Fout et al . ( 2017 ) ; Townshend et al . ( 2019 ) ; Liu et al . ( 2020 ) ) to benchmark each heteromer-compatible method . To expedite training and validation and to constrain memory usage , beginning with all remaining complexes not chosen for testing , we filtered out all complexes where either chain contains fewer than 20 residues and where the number of possible interface contacts is more than 2562 , leaving us with an intermediate total of 26,504 complexes for training and validation . In DIPS-Plus , binary protein complexes are grouped into shared directories according to whether they are derived from the same parent complex . As such , using a per-directory strategy , we randomly designate 80 % of these complexes for training and 20 % for validation to restrict overlap between our cross-validation datasets . After choosing these targets for testing , we then filter out complexes from our training and validation partitions of DIPS-Plus that contain any chain with over 30 % sequence identity to any chain in any complex in our test datasets . This threshold of 30 % sequence identity is commonly used in the bioinformatics literature ( Jordan et al . ( 2012 ) , Yang et al . ( 2013 ) ) to prevent large evolutionary overlap between a dataset ’ s cross-validation partitions . However , most existing works for interface contact prediction do not employ such filtering criteria , so the results reported in these works may be over-optimistic by nature . In performing such sequence-based filtering , we are left with 15,618 and 3,548 binary complexes for training and validation , respectively .

Summary: This paper proposed a novel geometry-evolving graph transformer for protein interface contact prediction, named DeepInteract. In particular, it predicts partner-specific protein interface contacts (i.e., inter-protein residue-residue contacts) given the 3D tertiary structures of two proteins as input, showcasing its effective use in learning representations of protein geometries to be exploited in downstream tasks. The experiments on challenging protein complex targets also demonstrated the proposed method achieves SOTA results for interface contact prediction.