P ERSISTENT T OR - ALGEBRA BASED STACKING ENSEMBLE LEARNING (PTA-SEL) FOR PROTEIN - PROTEIN BINDING AFFINITY PREDICTION
A BSTRACT
Protein-protein interactions (PPIs) play crucial roles in almost all biological processes.
Recently, Data-driven machine learning models have shown great power in the analysis of PPIs.
However, efficient molecular representation and featurization are still key issues that hinder the performance of learning models.
Here, we propose persistent Tor-algebra (PTA) PTA-based molecular characterization and featurization, and PTA-based stacking ensemble learning (PTA-SEL) for PPI binding affinity prediction for the first time.
More specifically, the Vietoris-Rips complex is used to characterize the PPI structure and its persistent Tor algebra is computed to form the molecular descriptors
These descriptors then are fed into our stacking model to make the prediction
We systematically test our model on the two most commonly used datasets i.e. SKEMPI and AB Bind
It has been found that our model outperforms all the existing models as far as we know, which demonstrates the great power of our model.
I NTRODUCTION
Protein-protein interactions (PPIs) play an essential role in a wide range of biological processes and mechanisms, including cell metabolism, signaling, protein transport and immune system (Geng et al., 2019b; Gonzalez & Kann, 2012).
The understanding of PPIs, in particular PPIs upon mutations, is significant to various biomedical applications, including disease-associated mutation analysis, drug design, and therapeutic intervention (Geng et al., 2019b; Gonzalez & Kann, 2012).
Experimentally, various methods have been developed to determine the protein structures and compute the binding affinities and stabilities of PPI.
However, experimental studies are time-consuming and labor-intensive.
Various efficient computational methods and models have been proposed for the
PPI studies.
Particularly, lots of models have been developed for the evaluation of PPI binding affinity changes upon mutations ( ∆∆ G ).
With the ever-increasing PPI data, a great amount of datadriven learning models have been developed (Geng et al., 2019b; Shi et al., 2021), including mCSM (Rodrigues et al., 2019), ELASPIC (Strokach et al., 2021), BindProf (Brender & Zhang, 2015), MutaBind (Zhang et al., 2020), iSEE (Geng et al., 2019a),
MuPIPR (Zhou et al., 2020), ProAffiMuSeq (Jemimah et al., 2020), GeoPPI (Liu et al., 2021), etc.
Recently, advanced mathematics, in particular topological data analysis (TDA) (Edelsbrunner et al., 2002; Zomorodian & Carlsson, 2005), are used in molecular representation and featurization (Cang & Wei, 2017c;
Nguyen et al., 2020; Cang et al., 2018; Meng & Xia, 2021).
Their combination with learning models have achieved great success in various steps of drug design, including proteinligand binding affinity prediction (Cang & Wei, 2017c;b; Nguyen et al., 2017; Cang & Wei, 2018;
Nguyen & Wei, 2019), protein stability change upon mutation prediction (Cang & Wei, 2017a; Cang et al., 2018), toxicity prediction (Wu & Wei, 2018; Chen et al., 2021; Jiang et al., 2021), solvation free energy prediction (Wang et al., 2016; 2018), partition coefficient and aqueous solubility (Wu et al., 2018), binding pocket detection (Zhao et al., 2018), and drug discovery (Gao et al., 2020).
Outstanding performance has been consistently achieved in D3R Grand challenge (Nguyen et al., 2019b;c;a).
In particular, TopNetTree has demonstrated great power in predicting binding affinity changes upon mutations (Wang et al., 2020a).
It outperformed all existing models and provided great insights for the SARS-CoV-2 mutations (Chen et al., 2020; Wang et al., 2020b).
Similar to homology, Tor-algebra is another homotopy invariant.
Topologically, two simplicial complexes share the same Tor-algebra if they can deform to each other (Buchstaber & Pano, 2015).
Tor-algebra of a simplicial complex K can be seen as a combination of the reduced simplicial cohomology of its subcomplexes.
Tor-algebra is from the Tor functor and the Tor functor is one of the central concepts of homological algebra.
Tor-algebra is highly related to the moment-angle complex, the key concept in toric topology.
More specifically, for a simplicial complex K , its moment-angle complex can be constructed, and the Tor algebra of K is isomorphic to the integral cohomology of the moment-angle complex of K .
Here we propose persistent Tor-algebra (PTA), PTA-based molecular characterization and featurization, and PTA-based stacking ensemble learning (PTA-SEL) model for PPIs, for the first time.
The
Vietoris-Rips complex is used to characterize the protein-protein structure and its persistent Toralgebra is computed to form the input features for our stacking model.
More specifically, 72 persistent Tor-algebra features are fed into 72 1D convolutional neural network models separately.
Besides the topological features, some precalculated auxiliary features from molecular physical properties are combined with the 72 topological features to form another 72 features, and these features are inputs for 72 gradient boosting tree models.
Then all 144 predictions are fed into the meta learner to make the prediction.
Our model is systematically tested on the two most-commonly used datasets,
SKEMPI and AB-Bind datasets, for PPI binding affinity changes upon mutations.
It has been found that our model can outperform all existing models, as far as we know.
R ESULTS P ERSISTENT T OR - ALGEBRA BASED MOLECULAR REPRESENTATION AND FEATURIZATION
Molecular representation and featurization are of great importance for the analysis of molecular data from material, chemistry and biology.
Recently, simplicial complex has been used in molecular representation, especially in drug design, the derived persistent homology theory has shown great power and is being hotly studied.
A simplicial complex is a set of vertices, edges, triangles and higher dimensional counterparts which are glued together along their faces.
Physically, a vertice can represent a molecular atom, residue, or even the whole molecule.
An edge can represent the interactions of various kinds between two vertices including covalent bonds, electrostatic, and other non-covalent forces.
The triangles, tetrahedrons and other higher dimensional counterparts can represent many-body interactions among several vertices, which characterize the higher dimensional structures of molecules.
Given a simplicial complex K with vertex set { v 1 , v 2 , ..., v m } , its face ring (or the Stanley-Reisner ring)
F [ K ] over the coefficient field F can be naturally constructed.
The face ring F [ K ] has the Fvector space basis consisting of monomials v α 1 j 1 ...v α i j i ...v α k j k where α
i = 1 , 2 , ... and { v j 1 , ..., v j k } is a simplex of K .
The face ring can uniquely determine its underlying simplicial complex while the face ring is an algebraic object and simplicial complex is a topological object, which means they are equivalent although they are different types of objects.
For the face ring F [ K ] of simplicial complex K , a Tor module Tor F [ m ] ( F [ K ] , F ) can be derived where F [ m ] is the polynomial algebra F [ v 1 , v 2 , ..., v m ] with degree 2 for each v i .
This is called the Toralgebra of the simplicial complex K .
It is a homotopy invariant so two simplicial complexes share the same Tor-algebra if they can deform to each other.
For simplicity, we denote Tor F [ m ] ( F [ K ] , F ) as
Tor ( K ) .
The Tor-algebra Tor ( K ) is highly related to simplicial cohomology of K .
Formally, there is a decomposition Tor ( K ) =
(cid:76) i,j> =
Tor − i, 2 j ( K ) where Tor − i, 2 j ( K ) is the ( − i, 2 j ) -grade component of Tor ( K ) .
And the simplicial cohomology of the full subcomplexes with j vertices of
K fully determine the ( − i, 2 j ) -grade component Tor − i, 2 j ( K ) .
More specifically, Tor − i, 2 j ( K ) =
(cid:76) J ⊂ [ m ] , | J | =
j ˆ H j − i − 1 ( K J , F ) where [ m ] is the vertex set { v 1 , ..., v m } , J is a vertex subset, K J is a full subcomplex of K obtained by restricting to J ⊂ [ m ] and ˆ H j − i − 1 ( K J ) is the reduced simplicial cohomology of K J .
From the decomposition, the Tor-algebra of a simplicial complex can be seen as a combination of the reduced simplicial cohomology of its subcomplexes.
Here, we propose persistent Tor-algebra, for the first time, through the combination of Tor-algebra and the filtration process.
More specifically, assume we have a filtration of simplicial complex.
That is a sequence of nested simplicial complexes connected by inclusions
[Equation]
where K i is a subcomplex of K i +1 .
For each simplicial complex K i , its face ring F ( K i ) over coefficient F can be constructed, which can uniquely determine its underlying simplicial complex.
Hence a series of face rings can be derived
[Equation] where two adjacent face rings are connected by the homomorphism induced from the inclusion map. Also, for each face ring F ( K i ) , its Tor-algebra Tor( K i ) can be constructed and the homomorphism naturally induce homomorphism between the Tor-algebra of two adjacent face rings.
Consequently, we get a sequence of Tor-algebra connected by homomorphisms
[Equation]
We call this sequence of Tor-algebra together with the homomorphisms as the persistent Tor-algebra.
Further, by considering a specific ( − i, 2 j ) -th graded component of Tor-algebra modules, we have
[Equation]
This is called the persistent Tor-algebra of K in the ( − i, 2 j ) -th graded component.
These Tor-algebra form a persistent module that can be represented as a persistent barcode or persistent diagram.
The bars in persistent barcode and the points in persistent diagram reflect the birth, death and evolution process of the Tor-algebra through the filtration process.
2. P ERSISTENT T OR ALGEBRA BASED STACKING MODELS FOR PROTEIN - PROTEIN BINDING AFFINITY PREDICTION 2.1. P ERSISTENT T OR - ALGEBRA BASED STACKING MODELS
Our Tor-algebra based stacking models consist of two types of base learners, 1D convolutional neural network and gradient boosting tree, and two meta learners, which can be seen in Figure 1.
The first type base learners consist of 72 1D CNN models with 72 types of Tor-algebra features as inputs separately.
Besides the topological features, some precalculated auxiliary features are combined with these topological features to form another 72 features, and these features are inputs of 72 gradient boosting tree models, the second type base learners.
We stack all the 144 base learners with a gradient boosting tree model and denote the model as
PTA-SEL(M2).
We also studied another stacking model PTA-SEL(M1) which only considers the first type base learners (1D CNN models).
2.1. F EATURE GENERATION
Protein-protein complexes are usually of great sizes, our aim is to predict the binding affinity change (∆∆ G ) following mutations.
And the mutation sites are very small, usually no more thanresidues.
So only protein atoms near mutation sites are considered to reduce computational cost.
More specifically, for each protein-protein complex, protein atoms within 10 ˚A of the mutation sites are considered.
We use the element-specific representations, six atom combinations are extracted, including { C } , { N } , { O } , { C, N } , { C, O } , and { N, O } .
Both the wild type and mutated type of the protein structure are considered.
So there are totally 12 atom combinations for each proteinprotein complex.
For each atom combination, a Vietoris-Rips complex K is constructed from the atom coordinates.
Then six persistent Tor-algebra components are computed, the indexes are (1 , 2) , (2 , 3) , ( n − 1 , 1) , ( n − 2 , n ) , ( n − 2 , n − 1) and ( n − 3 , n − 1) as ( i, j ) for Tor − i, 2 j ( K ) where n is the vertex number of K .
So totally 72 = 12 × 6 persistent Tor-algebra components are generated for each protein-protein complex.
These persistent Tor-algebra can be represented as persistent barcode.
Barcode statistic is used to discretize these barcodes into feature vectors.
More specifically, the filtration region [0 ˚A,10 ˚A] is divided into 40 equal-sized bins with grid size 0.25 ˚A.
For the ( i, j ) values of (1 , 2) , (2 , 3) , ( n − 1 , n ) and ( n − 2 , n − 1) , the values of the right endpoints of the bars in each bin are considered.
And for ( i, j ) values of ( n − 2 , n ) and ( n − 3 , n − 1) , the values of both the left and right endpoints of the bars in each bin are considered.
So 40 sets of real values are generated for each persistent Tor-algebra component with a specific ( i, j ) index.
Then six statistic values, including maximal value, minimal value, average value, sum, standard deviation and the number of elements are considered.
Hence a feature of 240 = 40 × 6 columns are generated for each persistent Tor-algebra with a specific ( i, j ) index.
In all, for a protein-protein complex, a totally 72 topological features of size 240 are generated.
Besides these topological features, based on Wang et al. (2020a), 707 auxiliary features are also considered in our model.
2.1. P ERFORMANCE
Two most commonly used datasets, AB-Bind S645 and SKEMPI S
(Pires & Ascher, 2016;
Wang et al., 2020a; Xiong et al., 2017), are considered in our benchmark tests.
In the performance assessment of our model, Pearson correlation coefficient (PCC) and root-mean-square error (RMSE) are used to assess the quality of prediction.
Ten independent regressions are performed and the average PCC and RMSE are used as the measurement of the performance of our model.
Figure 2 shows the comparison of performance between our model and other existing models on
AB-Bind S645 and SKEMPI S1131 datasets.
It can be seen that our model can achieve the best result with PCC of 0.866 and RMSE of 1.216 kcal/mol on SKEMPI S1131 and rank second on ABBind S645.
There are 27 nonbinders that do not follow the general distribution of the other data in the AB-Bind dataset.
It has been reported that these nonbinders have a strong negative impact on the prediction model accuracy (Wang et al., 2020a).
Our model can rank as first if we exclude thesenonbinders from the dataset.
More specifically, the PCC increases from 0.61 to 0.72 by excluding these 27 nonbinders.
Note that our PTA-SEL(M1) model also achieves state-of-the-art results, which demonstrates the great power of our persistent Tor-algebra theory in molecular representation and featurization.
C ONCLUSION
In this paper, we propose persistent Tor-algebra (PTA), PTA-based molecular representation and featurization, PTA-based stacking ensemble learning (PTA-SEL) models for PPI binding affinity prediction, for the first time.
In our stacking model, the Vietoris-Rips complex is used to characterize the PPI structure, then its persistent Tor-algebra features are computed.
These topological features are inputs of 72 1D CNN base models, and together with some auxiliary features form the inputs of 72 GBT base models.
Then all the 144 predictions of the 144 base learners are fed into a meta learner to make the prediction.
It is found that our model outperforms all the existing models for the protein-protein binding affinity prediction on both SKEMPI S1131 and AB-Bind S645 datasets, as far as we know.
A A PPENDIX A. P ERSISTENT T OR - ALGEBRA
In this section, we give construction of persistent Tor-algebra for a simplicial complex.
More specifically, we firstly give algebraic construction of Tor for any two modules.
Then we turn to the simplicial complex, the face ring is given and its associated Tor is defined as the Tor-algebra of the simplicial complex, more detailed descriptions can be found refer to (Buchstaber & Pano, 2015).
By considering a filtration process of the simplicial complex, we propose the persistent Tor-algebra for simplicial complex.
T OR MODULES
Assume A is a commutative finitely generated F -algebra with unit, graded by nonnegative even numbers (i.e. A = (cid:76) i> =
A i ) and connected (i.e. A 0 = F ).
The basic example is the polynomial algebra F [ v 1 , v 2 , ..., v m ] with degree ( v i ) = 2 .
We also assume that all A -modules M are nonnegatively graded and finitely generated, and all module maps are degree-preserving.
Definition 1.
Given an A -module M, a free resolutoin of M is an exact sequence of free A -modules
[Equation]
Here exact means Kerd i
= Imd i +1 (0 ⩽ i ) .
The resolution can be converted into a bigraded Fvector space R =
(cid:76) i R − i =
(cid:76) i,j R − i,j where R − i,j is the j -th graded component of the module
R − i , and the ( i, j ) -th component of d acts as d i,j : R − i,j → R − i +1 ,j .
We refer to the first grading of
R as external; it comes from the indexing of the terms in the resolution and is therefore nonpositive by our convention.
The second, internal, grading of R comes from the grading in the modules R − i and is therefore even and nonnegative.
Now we give the construction of Tor.
Assume we have a free resolution of an A -module M , and
N is another A -module.
Applying the functor (cid:78) A N to the free resolution, we obtain a cochain complex
[Equation]
The i -th cohomology module of the cochain complex is denoted as Tor iA ( M, N ) .
We can also write
[Equation]
where Tor − i, 2 j A ( M, N ) is the 2 j -th graded component of Tor − iA ( M, N ) .
Actually, for any Amodule M , there is a canonical way to construct a free resolution for M .
So the Tor A ( M, N ) can be defined for any A -module M .
A.1. F ACE RING OF A SIMPLICIAL COMPLEX
Definition 2.
Given a simplicial complex K on the vertex set { v 1 , ..., v m } , the face ring of K is the quotient graded ring
[Equation]
where I K =
( V I | I / ∈ K ) is the ideal generated by those monomials
V I for which I is not a simplex of K .
(For any vertex subset I = { v j 1 , ..., v j k } , V I is the monomial v j 1 ...v j k ).
The ideal I K is known as the Stanley − Reisner ideal of K .
Next we give a more clear description of the face ring.
Theorem 1.
Given a simplicial complex K , its face ring F [ K ] has the F -vector space basis consisting of monomials v α 1 j 1 v α 2 j 2 ...v α k j k where α i > 0 and { j i , j 2 , ..., j k } ∈ K .
Actually, the face ring determines its underlying simplicial complex.
Theorem 2 (Bruns-Gubeladze) .
Let F be a field, and K 1 , K 2 be two simplicial complexes on the vertex sets [ m 1 ] , [ m 2 ] respectively.
Suppose F [ K 1 ] and F [ K 2 ] are isomorphic as F -agebra.
Then there exists a bijective map [ m 1 ] → [ m 2 ] which induces an isomorphism between K 1 and K 2 .
A.
. T OR - ALGEBRA OF THE SIMPLICIAL COMPLEX
Given a simplicial complex K , its face ring F [ K ] has a F [ v 1 , v 2 , ..., v m ] -module structure via the quotient projection F [ v 1 , v 2 , ..., v m ]
→ F [ K ] .
Then its Tor-modules can be considered, we have
[Equation]
where F [ m ] =
F [ v 1 , v 2 , ..., v m ] and Tor − i, 2 j F [ m ] ( F [ K ] , F ) is 2 j -th graded component of
Tor i F [ m ] ( F [ K ] , F ) .
Definition 3.
Given a simplicial complex K , its face ring is denoted as F [ K ] .
The Tor-algebra of K is defined as Tor F [ v 1 ,v 2 ,...,v m ] ( F [ K ] , F ) .
The bigraded Betti number of F [ K ] is defined as
[Equation]
For simplicity, we denote the Tor-algebra of K as Tor ( K ) =
(cid:76) i,j> =
Tor − i, 2 j ( K ) .
The following fundamental result of Hochster reduces the calculation of the Betti numbers β − i, 2 j to the calculation of reduced simplicial cohomology of full subcomplexes in K .
Theorem 3 (Hochster) .
Given a simplicial complex K , its face ring is denoted as F [ K ] .
We have
[Equation]
where K J is a full subcomplex of K obtained by restricting to J ⊂ [ m ] and ˆ H ( K J , F ) is the reduced simplicial cohomology of K J .
We assume ˆ H − 1 = F .
The Tor-algebra is highly related to a famous concept, moment-angle complex, which is also the central concept in toric topology.
For a simplicial complex K , the moment-angle complex Z K can be constructed.
Actually, the integral cohomology ring of the moment-angle complex Z K is isomorphic to the Tor-algebra of the simplicial complex K .
A.1. P ERSISTENT T OR - ALGEBRA OF THE SIMPLICIAL COMPLEX
Now we construct the persistent Tor-algebra for a simplicial complex.
Assume we have a filtration of simplicial complex K .
That is, a sequence of nested simplicial complexes:
[Equation]
where each K i is a subcomplex of K i +1 .
We consider the inclusion map from K i to K
i +1 , this simplicial map naturally induces a homomorphism from the face ring of K i to the face ring of K
i +1 .
Hence we get a sequence of face rings connected by homomorphisms
[Equation]
For each face ring F ( K i ) , its Tor-algebra can be generated, so we get a sequence of Tor-algebra
[Equation]
where two adjacent Tor-algebras are connected by homomorphisms induced from the homomorphisms of face rings.
We call this sequence of Tor-algebra modules the persistent Tor-algebra of the filtration.
Further, by considering a specific graded component of Tor-algebra modules, we can get the persistent graded Tor-algebra for a given ( − i, 2 j ) .
We have
[Equation]
This is called the persistent Tor-algebra of K in the ( − i, 2 j ) -th graded component.
Similar to persistent homoloy, these Tor-algebra form a persistent module so that they can be represented as a persistent barcode or persistent diagram where the bars in the persistent barcode and points in the persistent diagram reflect the birth, death and evolution of the Tor-algebra through the filtration process.
A. P ERSISTENT T OR - ALGEBRA BASED STACKING MODEL
Our Tor-algebra based stacking model consists of several GBT models and 1D CNN models.
Computationally, all the 72 1D CNN models have the same architecture and hyperparameters, and all the 72 GBT models also share the same parameters.
The detailed parameter setting of GBT is illustrated in Table 1.
The general architecture for 1D CNN is demonstrated in Figure 3.
The CNN [Table] hyperparameters are as follows:
• ReLU on all 4 convolutional layers.
• Dropout(0.3)
• First layer weight initialised by he normal.
• Rest of weights initialised by lecun uniform • Batchsize=16, 2000 epochs.
• Adam Optimizer with lr=1e-5.
• ReLU on fully connected layers.
A. P ERFORMANCE
Tenfold corss validation is used to do the regression.
Pearson correlation coefficient (PCC) and rootmean-square error (RMSE) are used to assess the quality of prediction.
Ten independent regressions are performed and the average PCC and RMSE are used as the measurement of the performance of our model.
The detailed performance between our models and other existing models on SKEMPI
S1131 and AB-Bind S645 can be found in Table 2 and Table 3 respectively [Table] [Table]
Discovery
[Table]