BACKGROUND
the knowledge of the interactions among proteins is essential to understanding the molecular mechanisms inside the cell.
however, the experimental measurement of protein-protein interactions by two main procedures--the yeast two-hybrid system and mass spectrometry combined with tandem affinity purification--suffers from a high false positive rate, as evidenced by the small intersection between several independently generated interaction data sets  <cit> .
recent years have seen much progress in understanding of the false positive predictions  <cit> .
the limitations of current experimental methods therefore highlight the need for in silico interaction predictions.
the elucidation of an increasing number of protein 3d structures is likely to continue at a fast pace as a result of several large-scale initiatives.
these structures provide both an opportunity and a challenge for in silico prediction methods.
the opportunity is that if in silico methods are able to predict whether two given 3d structures interact, then these methods may be applied to predict interactions among the large amount of proteins with known or inferred 3d structure.
the challenge is that these methods need to be computationally fast to scale to the prediction of interactions among millions or more candidate pairs of structures.
unfortunately, the current methods of choice to predict interactions are mostly based on the idea of docking, which is very computationally intensive and therefore unlikely to be able to scale to large interactomes in the near future.
see  <cit>  for a review of the issues related to the prediction of interaction using protein-protein docking, and  <cit>  for a review of the problem of predicting interactions using structural information.
in this study, we are interested in developing methods to predict whether two proteins interact.
we aim to develop a method that will scale to the analysis of whole proteomes in order to identify candidate protein pairs for further analysis by more computationally expensive procedures.
docking not only attempts to answer the question of whether two proteins interact, but also how they interact.
our strategy is to develop methods that predict less information than docking, by focusing only on the first question, trading this decrease in the richness of prediction for an increase in computational efficiency.
note that, in general, the problem of predicting protein-protein interactions is complicated by multi-domain proteins.
for simplicity in this work, we will therefore focus on predicting whether a given pair of protein domains interacts.
we aim at answering the question: "given two domain structures, do they interact?"
our strategy to solve this problem is to formalize it as a binary classification problem, and to train a discriminative classifier to answer the question of whether or not a pair of domain structures interact, using as training data examples of known interacting and non-interacting pairs.
this idea is related to previous applications of machine learning techniques to predict protein-protein interactions from a variety of data types, including noisy interaction networks, localization information, sequence and expression data.
these techniques include bayesian networks  <cit> , support vector machines  <cit> , decision trees  <cit>  and random forests  <cit> .
shoemaker et al.
<cit>  review the approaches for predicting protein-protein interactions, and  <cit>  compare existing approaches on a common data set.
however, to our knowledge, machine learning methods have not previously been applied to the prediction of protein-protein interactions from protein structures.
¿from a technical point of view, we face two issues to implement this strategy.
first, we must be able to manipulate protein 3d structures in the context of a machine learning algorithm.
for this we borrow an idea from  <cit> , who proposed using a classical measure of structural similarity to implicitly embed 3d structures in a vector space.
second, once we represent each 3d structure as a vector, we must be able to learn a classification function over pairs of vectors.
for this we test different strategies that have been proposed recently to solve this issue with kernel methods such as svms  <cit> .
combining these two ideas, we propose two learning algorithms that can be combined with two possible representations of pairs of 3d structures to predict interactions, resulting in four methods.
in order to assess the benefits of using 3d structure information for this purpose, rather than using sequence information only, we additionally test the same four methods starting from a measure of primary sequence similarity, instead of 3d structure similarity.
in order to test these methods, we construct three benchmarks from known interactions between proteins of known or inferred structures.
we compare the performance of the different methods in a cross-validation procedure.
as a baseline, we compare to similar methods that exploit only amino acid sequences.
these experiments suggest that the metric learning pairwise kernel  <cit> , coupled with a support vector machine classifier, yields the best performance.
methods
our approach to predict whether two proteins interact is to frame the question as a supervised learning problem.
based on examples of interacting pairs and non-interacting pairs, we train a binary classifier to predict the class  of a pair of protein structures.
in this study, we compare eight different classification methods, corresponding to all combinations with respect to three binary choices:  two classification algorithms,  two vector encodings of proteins, based on structure or sequence, respectively, and  two different methods for computing the similarities among pairs of vectors.
in the following, we first describe the two classification algorithms, the two similarity functions, and then the two methods for computing pairwise similarities.
classification algorithms
nearest neighbor
as a simple baseline algorithm, we use the nearest neighbor algorithm .
given a training set of n points x <dig> ...,xn labeled y <dig> ...,yn in {- <dig>  +1}, nn assigns to a test point  the class label of the majority among the k nearest training set points.
in our case, rather than a simple binary label, we would like the algorithm to return a real-valued discriminant score in order to be able to rank the prediction by confidence.
for this purpose, we use the difference in the sum of the distances to the k nearest positive and the sum of distances to the k nearest negative training set examples:
the predicted class label is simply the sign of this discriminant.
denotes the set of k nearest negative  neighbors.
the value of k is selected via cross-validation, as described in the results section.
nn is perhaps the simplest example of a broad class of algorithms known collectively as kernel methods  <cit> .
an algorithm is a kernel method if and only if it can be formulated such that every occurrence of a data vector x occurs inside of a scalar product operation.
in this case, every instance of the scalar product operation can be replaced by a generalized notion of similarity, called the kernel function.
formally, a kernel function is a symmetric positive semidefinite function, which provably corresponds to the scalar product operation in some vector space.
using a kernel function in place of the scalar product operation implicitly maps the given data set into a possibly high-dimensional space  that may be non-linearly related to the input space in which the data resides.
to "kernelize" the nearest neighbor algorithm, we simply note that the euclidean distance can be stated entirely in terms of scalar product operations:
in terms of the kernel function k = ⟨;Φ(), Φ⟩;, therefore,
the kernel nn algorithm simply substitutes equation  into equation .
below, we define three different kernel functions and use each of them to create a nn classifier for predicting protein-protein interactions.
support vector machine
functionally, the support vector machine   <cit>  is similar to the nearest neighbor algorithm.
both are supervised classification algorithms, and both are kernel methods.
however, unlike the nearest neighbor approach, the svm algorithm attempts to find a hyperplane that optimally separates examples from the two given classes.
the svm algorithm is appealing due to its mathematical properties and its successful application to a wide variety of classification problems in computational biology  <cit> .
training an svm involves solving a convex quadratic program, which can be formulated as follows:
subject to the constraints
the solution to this optimization is a hyperplane hw,b = {x: ⟨w, Φ⟩ + b = 0}, and the classification procedure for a novel test point  involves computing which side of the hyperplane the point lies on.
the svm discriminant is the signed distance to the hyperplane:
like nn, the svm can also be subjected to the "kernel trick," the details of which we omit here for brevity.
the svm depends on a user-specified regularization term, c ≥  <dig>  which prevents overfitting to noise in the data or noise in the labels.
in the case of unbalanced data, we apply this regularization asymmetrically to positive and negative examples, as follows:
the values of c+ and c- are selected via cross-validation, as described in the results section.
kernel on proteins
in order to use a nn or svm algorithm to predict interactions between proteins, we must define a kernel between the objects for which predictions are to be made, namely, pairs of proteins.
in this section, we describe two kernels that are defined with respect to individual proteins.
in the next section, we describe methods for generalizing from these single-protein kernels to kernels on protein pairs.
a kernel on protein structures
a natural place to begin, when considering comparisons among protein structures, is with structural alignment algorithms such as ce  <cit> , dali  <cit>  and mammoth  <cit> .
these algorithms create an alignment between two proteins and then compute a score that reflects the alignment's quality.
in this work, we use mammoth  <cit> , which is efficient and produces high quality alignments.
we treat the output of mammoth as an arbitrary score, denoted s.
unfortunately, the alignment quality score returned by mammoth cannot be used as a kernel function directly, because the score is not positive semidefinite .
to define a mammoth kernel on protein structures, we therefore subtract the negative portion of the eigenvalue spectrum to convert the score to a kernel.
namely, if m is the mammoth similarity matrix, then m is symmetric with singular value decomposition
where d is a diagonal matrix diag.
the mammoth kernel is then
with Ψ = diag,...,ψ), and ψ =  <dig> + λ if λ >  <dig>  and  <dig> otherwise.
in practice, we normalize this kernel by projecting onto the unit sphere, via the transformation
a related mammoth kernel has previously been shown to yield good performance in classifying proteins into scop categories or by go terms  <cit> .
it uses ψ = λ <dig>
a kernel on protein sequences
we compare our methods based on structures to corresponding sequence-based methods, as a baseline.
we borrow an idea of a kernel on sequences from  <cit> .
the mismatch kernel induces a distance measure between protein sequences.
a protein sequence α is mapped to a feature vector
where  is the alphabet of  <dig> amino acids.
the neighbourhood k,m of a k-mer α is the set of k-mers that differs in at most m positions.
the feature vector encodes the all the k-mers in the neighborhood:
thus, the mismatch kernel between two protein sequences x and y is
in our experiments, we set k =  <dig> and m =  <dig>
kernels for pairs of objects
this section describes techniques for deriving, from the single-protein kernels described in the previous section, a pairwise kernel function k, ) that quantifies the degree of similarity between two protein pairs.
in particular, we present two pairwise kernels: the tensor product pairwise kernel  and the metric learning pairwise kernel .
tensor product pairwise kernel
the tensor product pairwise kernel  is a general method for building a kernel for pairs of objects from any kernel k for objects.
this kernel has been used successfully for predicting protein-protein interactions from sequence  <cit>  and from a combination of sequence, annotation, network topology and interolog features  <cit> .
the tppk considers two pairs of proteins to be similar to one another when each protein from one pair is similar to one protein of the other pair.
for example, if p <dig> is similar to q <dig> and p <dig> is similar to q <dig>  then we can say that the pairs  and  are similar to one another.
we can translate these intuitions into the following pairwise kernel:
where k is the kernel on proteins.
the feature space for tppk is the tensor product of the feature spaces of each member of the pair, so if m is the dimensionality of the feature space, then the dimensionality of the feature space of the tensor product is m <dig>
note that, as formulated in equation , the tppk has the counter-intuitive property that two protein pairs can be considered similar if the underlying protein pairs are strongly dissimilar.
this is because the base kernel function k can return negative values which, when multiplied together, yield a positive value for ktppk.
to avoid this artifact, we add  <dig> to the base kernel function, so that the kernel values lie in the range  <cit> .
this operation preserves a valid kernel, because the matrix of all 1's is positive definite, and the sum of two positive definite matrices is a positive definite matrix.
without this "add 1" correction, the tppk results are significantly worse .
metric learning pairwise kernel
similar to tppk, the metric learning pairwise kernel  is a method for building a kernel for pairs of objects based on a kernel k for objects  <cit> .
however, mlpk represents a pair of objects as the difference between its members.
in this case, a pair  of objects is represented in the feature space by the difference p <dig> - p <dig>  squared  to make it invariant with respect to the order of the proteins.
the mlpk was presented in  <cit>  as a way to learn a new metric between individual objects .
the corresponding mlpk between the pair  and  is
for consistency with the tppk, we add  <dig> to each entry of k.
data
we built a benchmark of interacting pairs of protein structures, based on the database of interacting proteins   <cit>  downloaded on january  <dig>   <dig>  the complete database contains  <dig>  interactions among  <dig>  proteins.
from these interactions, we selected two subsets.
the first is the "core" dataset, which consists of a set of  <dig>  interactions among  <dig>  s. cerevisiae proteins.
these interactions are considered reliable based on expression data and the presence of paralogous interacting protein pairs  <cit> .
the second data set, "small-scale," consists of interactions that were verified by small-scale experimental methods, using techniques that reliably indicate direct physical interaction of proteins.
these methods include immunoprecipitation, cross-linking, in vitro binding, x-ray diffraction, competition binding, electron microscopy and x-ray crystallography, for a total of  <dig>  interactions among  <dig>  proteins.
for each of the two sets of interactions, we collected a set of protein structures using the following protocol:
<dig>  we use psi-blast to search for homologous proteins with known structures, as shown in figure  <dig>  for each protein in a given dip data set, we search the domains of the pdb using psi-blast for a maximum of  <dig> iterations  <cit> .
for each protein, we identify at most one homolog that is longer than  <dig> amino acids and achieves an e-value less than 10- <dig>
<dig>  all of our methods require negative examples for training.
to avoid biasing the selection of negatives, we choose non-interacting protein pairs at random from a non-redundant subset of the pdb  <cit> .
this subset consists of  <dig>  structures, selected using the pisces database with a percent identify cutoff of 70% and a resolution of  <dig>  Å or better .
at this stage, we select five times as many negative pairs as positive pairs.
<dig>  we filter the combined collection of positive and negative pairs to remove redundant pairs.
for each data set, we cluster the protein sequences using blastclust  <cit>  with a threshold of 40% sequence identity.
we search for cases in which pairs of proteins link the same two clusters.
if all pairs are negative, then we remove all except one randomly selected pair.
if one pair is positive and others are negative, then we remove the negative pairs.
if two or more pairs are positive, then we remove all the pairs except the positive one that has the minimal number of  neighbors.
<dig>  finally, we remove all negative pairs that involve a protein shorter than  <dig> amino acids and then randomly downsample the remaining negative pairs so that the ratio of negative to positive pairs is 3: <dig>
this procedure generated two different data sets.
the "core" data set contains  <dig>  proteins, with  <dig>  interacting pairs involving  <dig> proteins and  <dig>  non-interacting pairs.
the "small-scale" set contains  <dig>  proteins, with  <dig>  interacting pairs involving  <dig>  proteins and  <dig>  non-interacting pairs.
we also build a subset of the "core", called "core-subset," that contains the same number of interacting pairs as the small-scale.
note that we also generated three additional "core" and three additional "small-scale" data sets, using homology thresholds of 10-20and 10- <dig> and redundancy thresholds of 40% and 90%.
the sizes and cross-validation results from those data sets are summarized in the additional file  <dig>
all data sets used in this study are available at http://noble.gs.washington.edu/proj/pips.
RESULTS
for each of the eight classifiers, we performed 5-fold cross-validation, repeated three times .
during each svm training, an internal 5-fold cross-validation was performed to select two regularization parameters using grid search.
we selected the regularization parameter associated with interacting pairs  from {10- <dig>  10- <dig> ..., 108}, and we selected the ratio c+/c- from { <dig>   <dig>  100}.
the parameter k of nn was selected from { <dig>   <dig>   <dig>   <dig>   <dig>  15}.
in each testing phase of the cross-validation, the proportion of negative examples with respect to positive examples is equal to r =  <dig>  in a more realistic setting, this ratio would be much higher.
therefore, we extrapolate from the measured precision p to estimate the precision of each classifier in the case where all negative examples were considered in the benchmark.
if v is the number of proteins, and e is the number of interacting pairs, then the ratio of negative to positive examples is r' = / <dig> - e)/e.
the estimated precision thus becomes p/).
to evaluate the statistical significance of the differences in performance observed in the top row of figure  <dig>  we performed wilcoxon signed-rank tests on the area under the precision-recall curve across the  <dig> splits of the 3×5cv of the four data sets.
the bottom row of panels in figure  <dig> shows the results of this test.
in the graph, an edge from method a to b indicates that method a outperforms method b according to the wilcoxon test with p <  <dig> .
redundant edges have been removed for clarity; i.e., the figure shows the transitive reduction of the full graph.
the signed-rank results show two expected trends across the three benchmarks.
first, in general, structure-based methods perform better than the corresponding sequence-based methods.
in particular, classifiers that use a mammoth kernel outperform the corresponding mismatch kernel method in  <dig> cases out of  <dig>  and never perform worse.
second, svm-based methods generally perform better than nearest neighbors methods, although this trend is not as consistent .
the relative performance of the svm and nn classifiers seems to depend in part upon which pairwise transformation is employed.
in conjunction with the mlpk transformation, the svm outperforms the nn in all six cases.
in contrast, when the tppk transformation is employed, nn performs better than the svm in four cases, and ties in the remaining two.
although the relative performance of the mlpk and tppk transformations is difficult to explain  <cit> , one may speculate that the tppk definition  results in a more natural measure of similarity between pairs than the mlpk one ; hence, tppk is likely to behave better with a nn classifier.
on the other hand, the mlpk transform can be justified from the point of view of metric learning when used in combination with an svm  <cit> , but mlpk remains an obscure measure of similarity between pairs, which may therefore not be optimally used by a nn classifier.
finally, with respect to mlpk versus tppk, the mlpk transformation performs best overall.
in  <dig> out of  <dig> cases, the mlpk method outperforms the corresponding tppk method.
the only three exceptions are nn classifiers using sequence kernels, which are relatively poorly performing methods overall.
overall, our experiments suggest that, for the prediction of protein-protein interactions from structure, the mlpk svm combined with mammoth is the best method among the eight that we considered.
this method performs best on all three benchmarks.
having selected the best performing method, we used an svm to predict novel interactions among all pairs of proteins in a non-redundant subset of the pdb.
we used the pisces database with a percent identify cutoff of 90% and a resolution of  <dig>  Å or better , eliminating proteins shorter than  <dig> amino acids and longer than  <dig> amino acids.
this set contains  <dig>  structures.
prior to the svm analysis, we eliminated pairs that are similar to any member of the training set.
specifically, for a candidate pair , we define the distance to a training set pair  as
where e is the blast e-value assigned to query p against target q. we only consider candidate pairs for which the minimum distance to any member of the training set exceeds an e-value of  <dig> .
this filtering reduces the complete set from  <dig> , <dig> to  <dig> , <dig> pairs.
we applied to each of these pairs the svm mlpk mammoth method, trained on 90% of the small-scale dip benchmark, and we used the held-out 10% of the data to convert the svm discriminants to probabilities  <cit> .
at a threshold of 99%, the svm predicts  <dig>  novel interactions; at 90%, the svm predicts  <dig>  pairs.
the latter set is publicly available via the yeast resource center public data repository http://yeastrc.org/pdr <cit> .
discussion
we presented eight machine learning methods for large-scale prediction of interactions between pairs of protein structures.
the methods use either a nn or a svm classifier, coupled with two kernels on protein sequences or structures, coupled with two methods for representing similarities between pairs of protein.
in our cross-validation experiment, we observed--not surprisingly--that structure-based methods are generally more accurate than the methods based on sequence.
among structure-based classifiers, the svm with an mlpk kernel performs best.
the classifiers described here only use structural or sequence information about the proteins.
further improvements in prediction accuracy could probably be obtained by taking into account other information such as expression profiles or sub-cellular localization.
the integration of such heterogeneous data could for example be carried out by creating different kernels from each type of information, and merging the information by forming a linear combination of all individual kernels  <cit> .
one shortcoming of our approach is that each protein interaction is predicted independently.
we believe that taking into account the topology of the whole interaction graph might improve the accuracy of the predictions.
however, generalizing binary prediction to more complex prediction, such as graph prediction, remains a challenging issue and an active research topic in the machine learning community.
in the course of this study we generated eight benchmarks that we make publicly available and which can be used to assess the performance of other methods for the automated prediction of interaction between protein structures.
the benchmark generation depends on two parameters: the homology threshold, to be input to psi-blast to find a proxy pair, and the redundancy threshold to be input to blastclust.
for each of the benchmarks, we set the values of the homology threshold to 10- <dig>  and 10- <dig>  and the values of the redundancy thresholds to  <dig> and  <dig>  leading to four datasets for each benchmark.
here, we reported the cross-validation results of only one dataset; however, the conclusions do not change when we consider the cross-validation experiments on the other benchmark data sets.
finally, as pointed out in the introduction, the main motivation behind this work is to perform a large-scale screening of pairs of proteins likely to interact, before validating the prediction by more expensive computational methods such as docking, or experimental methods for pairs of particular interest.
this will be the subject of future work.
CONCLUSIONS
the eight presented machine learning methods based on kernel methods to predict the binary interaction between protein structures were evaluated on a data set of reliable interations.
we draw three main conclusions from our experiments.
first, using structure we can more accurately predict interactions than using sequence alone.
second, svm methods compare favorably against the methods based on the nearest neighbor classifier.
third, the best choice of pairwise metric associated with the predictor is mlpk rather than tppk.
authors' contributions
mh, jpv and wsn designed the study and drafted the manuscript.
mh performed the experiments.
mr included the predicted interactions in the yrc database.
mh, mr, jpv and wsn read and approved the final manuscript.
supplementary material
additional file 1
cross-validation results on additional core and small-scale datasets.
the file contains the size and cross-validation results of eight datasets, including three additional core and three additional small-scale.
the core and small-scale datasets are parametrized by two parameters during generation, the homology threshold and the redundancy threshold.
the datasets correspond to two values for the homology threshold and two values of redundancy threshold.
click here for file
acknowledgements
this work was funded in part by nih award p <dig> rr <dig>

 the knowledge of the interactions among proteins is essential to understanding the molecular mechanisms inside the cell. however, the experimental measurement of protein-protein interactions by two main procedures--the yeast two-hybrid system and mass spectrometry combined with tandem affinity purification--suffers from a high false positive rate, as evidenced by the small intersection between several independently generated interaction data sets. recent years have seen much progress in understanding of the false positive predictions. the limitations of current experimental methods therefore highlight the need for in silico interaction predictions. the elucidation of an increasing number of protein 3d structures is likely to continue at a fast pace as a result of several large-scale initiatives. these structures provide both an opportunity and a challenge for in silico prediction methods. the opportunity is that if in silico methods are able to predict whether two given 3d structures interact, then these methods may be applied to predict interactions among the large amount of proteins with known or inferred 3d structure. the challenge is that these methods need to be computationally fast to scale to the prediction of interactions among millions or more candidate pairs of structures. unfortunately, the current methods of choice to predict interactions are mostly based on the idea of docking, which is very computationally intensive and therefore unlikely to be able to scale to large interactomes in the near future. see for a review of the issues related to the prediction of interaction using protein-protein docking, and for a review of the problem of predicting interactions using structural information. in this study, we are interested in developing methods to predict whether two proteins interact. we aim to develop a method that will scale to the analysis of whole proteomes in order to identify candidate protein pairs for further analysis by more computationally expensive procedures. docking not only attempts to answer the question of whether two proteins interact, but also how they interact. our strategy is to develop methods that predict less information than docking, by focusing only on the first question, trading this decrease in the richness of prediction for an increase in computational efficiency. note that, in general, the problem of predicting protein-protein interactions is complicated by multi-domain proteins. for simplicity in this work, we will therefore focus on predicting whether a given pair of protein domains interacts. we aim at answering the question: "given two domain structures, do they interact?" our strategy to solve this problem is to formalize it as a binary classification problem, and to train a discriminative classifier to answer the question of whether or not a pair of domain structures interact, using as training data examples of known interacting and non-interacting pairs. this idea is related to previous applications of machine learning techniques to predict protein-protein interactions from a variety of data types, including noisy interaction networks, localization information, sequence and expression data. these techniques include bayesian networks, support vector machines, decision trees and random forests. shoemaker et al. review the approaches for predicting protein-protein interactions, and compare existing approaches on a common data set. however, to our knowledge, machine learning methods have not previously been applied to the prediction of protein-protein interactions from protein structures. ¿from a technical point of view, we face two issues to implement this strategy. first, we must be able to manipulate protein 3d structures in the context of a machine learning algorithm. for this we borrow an idea from, who proposed using a classical measure of structural similarity to implicitly embed 3d structures in a vector space. second, once we represent each 3d structure as a vector, we must be able to learn a classification function over pairs of vectors. for this we test different strategies that have been proposed recently to solve this issue with kernel methods such as svms. combining these two ideas, we propose two learning algorithms that can be combined with two possible representations of pairs of 3d structures to predict interactions, resulting in four methods. in order to assess the benefits of using 3d structure information for this purpose, rather than using sequence information only, we additionally test the same four methods starting from a measure of primary sequence similarity, instead of 3d structure similarity. in order to test these methods, we construct three benchmarks from known interactions between proteins of known or inferred structures. we compare the performance of the different methods in a cross-validation procedure. as a baseline, we compare to similar methods that exploit only amino acid sequences. these experiments suggest that the metric learning pairwise kernel, coupled with a support vector machine classifier, yields the best performance. methods our approach to predict whether two proteins interact is to frame the question as a supervised learning problem. based on examples of interacting pairs and non-interacting pairs, we train a binary classifier to predict the class of a pair of protein structures. in this study, we compare eight different classification methods, corresponding to all combinations with respect to three binary choices: two classification algorithms, two vector encodings of proteins, based on structure or sequence, respectively, and two different methods for computing the similarities among pairs of vectors. in the following, we first describe the two classification algorithms, the two similarity functions, and then the two methods for computing pairwise similarities. classification algorithms nearest neighbor as a simple baseline algorithm, we use the nearest neighbor algorithm. given a training set of n points x...,xn labeled y...,yn in {- +1}, nn assigns to a test point the class label of the majority among the k nearest training set points. in our case, rather than a simple binary label, we would like the algorithm to return a real-valued discriminant score in order to be able to rank the prediction by confidence. for this purpose, we use the difference in the sum of the distances to the k nearest positive and the sum of distances to the k nearest negative training set examples: the predicted class label is simply the sign of this discriminant. denotes the set of k nearest negative neighbors. the value of k is selected via cross-validation, as described in the results section. nn is perhaps the simplest example of a broad class of algorithms known collectively as kernel methods. an algorithm is a kernel method if and only if it can be formulated such that every occurrence of a data vector x occurs inside of a scalar product operation. in this case, every instance of the scalar product operation can be replaced by a generalized notion of similarity, called the kernel function. formally, a kernel function is a symmetric positive semidefinite function, which provably corresponds to the scalar product operation in some vector space. using a kernel function in place of the scalar product operation implicitly maps the given data set into a possibly high-dimensional space that may be non-linearly related to the input space in which the data resides. to "kernelize" the nearest neighbor algorithm, we simply note that the euclidean distance can be stated entirely in terms of scalar product operations: in terms of the kernel function k = ⟨;Φ(), Φ⟩;, therefore, the kernel nn algorithm simply substitutes equation into equation. below, we define three different kernel functions and use each of them to create a nn classifier for predicting protein-protein interactions. support vector machine functionally, the support vector machine is similar to the nearest neighbor algorithm. both are supervised classification algorithms, and both are kernel methods. however, unlike the nearest neighbor approach, the svm algorithm attempts to find a hyperplane that optimally separates examples from the two given classes. the svm algorithm is appealing due to its mathematical properties and its successful application to a wide variety of classification problems in computational biology. training an svm involves solving a convex quadratic program, which can be formulated as follows: subject to the constraints the solution to this optimization is a hyperplane hw,b = {x: ⟨w, Φ⟩ + b = 0}, and the classification procedure for a novel test point involves computing which side of the hyperplane the point lies on. the svm discriminant is the signed distance to the hyperplane: like nn, the svm can also be subjected to the "kernel trick," the details of which we omit here for brevity. the svm depends on a user-specified regularization term, c ≥ which prevents overfitting to noise in the data or noise in the labels. in the case of unbalanced data, we apply this regularization asymmetrically to positive and negative examples, as follows: the values of c+ and c- are selected via cross-validation, as described in the results section. kernel on proteins in order to use a nn or svm algorithm to predict interactions between proteins, we must define a kernel between the objects for which predictions are to be made, namely, pairs of proteins. in this section, we describe two kernels that are defined with respect to individual proteins. in the next section, we describe methods for generalizing from these single-protein kernels to kernels on protein pairs. a kernel on protein structures a natural place to begin, when considering comparisons among protein structures, is with structural alignment algorithms such as ce, dali and mammoth. these algorithms create an alignment between two proteins and then compute a score that reflects the alignment's quality. in this work, we use mammoth, which is efficient and produces high quality alignments. we treat the output of mammoth as an arbitrary score, denoted s. unfortunately, the alignment quality score returned by mammoth cannot be used as a kernel function directly, because the score is not positive semidefinite. to define a mammoth kernel on protein structures, we therefore subtract the negative portion of the eigenvalue spectrum to convert the score to a kernel. namely, if m is the mammoth similarity matrix, then m is symmetric with singular value decomposition where d is a diagonal matrix diag. the mammoth kernel is then with Ψ = diag,...,ψ), and ψ = + λ if λ > and otherwise. in practice, we normalize this kernel by projecting onto the unit sphere, via the transformation a related mammoth kernel has previously been shown to yield good performance in classifying proteins into scop categories or by go terms. it uses ψ = λ a kernel on protein sequences we compare our methods based on structures to corresponding sequence-based methods, as a baseline. we borrow an idea of a kernel on sequences from. the mismatch kernel induces a distance measure between protein sequences. a protein sequence α is mapped to a feature vector where is the alphabet of amino acids. the neighbourhood k,m of a k-mer α is the set of k-mers that differs in at most m positions. the feature vector encodes the all the k-mers in the neighborhood: thus, the mismatch kernel between two protein sequences x and y is in our experiments, we set k = and m = kernels for pairs of objects this section describes techniques for deriving, from the single-protein kernels described in the previous section, a pairwise kernel function k, ) that quantifies the degree of similarity between two protein pairs. in particular, we present two pairwise kernels: the tensor product pairwise kernel and the metric learning pairwise kernel. tensor product pairwise kernel the tensor product pairwise kernel is a general method for building a kernel for pairs of objects from any kernel k for objects. this kernel has been used successfully for predicting protein-protein interactions from sequence and from a combination of sequence, annotation, network topology and interolog features. the tppk considers two pairs of proteins to be similar to one another when each protein from one pair is similar to one protein of the other pair. for example, if p is similar to q and p is similar to q then we can say that the pairs and are similar to one another. we can translate these intuitions into the following pairwise kernel: where k is the kernel on proteins. the feature space for tppk is the tensor product of the feature spaces of each member of the pair, so if m is the dimensionality of the feature space, then the dimensionality of the feature space of the tensor product is m note that, as formulated in equation, the tppk has the counter-intuitive property that two protein pairs can be considered similar if the underlying protein pairs are strongly dissimilar. this is because the base kernel function k can return negative values which, when multiplied together, yield a positive value for ktppk. to avoid this artifact, we add to the base kernel function, so that the kernel values lie in the range. this operation preserves a valid kernel, because the matrix of all 1's is positive definite, and the sum of two positive definite matrices is a positive definite matrix. without this "add 1" correction, the tppk results are significantly worse. metric learning pairwise kernel similar to tppk, the metric learning pairwise kernel is a method for building a kernel for pairs of objects based on a kernel k for objects. however, mlpk represents a pair of objects as the difference between its members. in this case, a pair of objects is represented in the feature space by the difference p - p squared to make it invariant with respect to the order of the proteins. the mlpk was presented in as a way to learn a new metric between individual objects. the corresponding mlpk between the pair and is for consistency with the tppk, we add to each entry of k. data we built a benchmark of interacting pairs of protein structures, based on the database of interacting proteins downloaded on january the complete database contains interactions among proteins. from these interactions, we selected two subsets. the first is the "core" dataset, which consists of a set of interactions among s. cerevisiae proteins. these interactions are considered reliable based on expression data and the presence of paralogous interacting protein pairs. the second data set, "small-scale," consists of interactions that were verified by small-scale experimental methods, using techniques that reliably indicate direct physical interaction of proteins. these methods include immunoprecipitation, cross-linking, in vitro binding, x-ray diffraction, competition binding, electron microscopy and x-ray crystallography, for a total of interactions among proteins. for each of the two sets of interactions, we collected a set of protein structures using the following protocol: we use psi-blast to search for homologous proteins with known structures, as shown in figure for each protein in a given dip data set, we search the domains of the pdb using psi-blast for a maximum of iterations. for each protein, we identify at most one homolog that is longer than amino acids and achieves an e-value less than 10- all of our methods require negative examples for training. to avoid biasing the selection of negatives, we choose non-interacting protein pairs at random from a non-redundant subset of the pdb. this subset consists of structures, selected using the pisces database with a percent identify cutoff of 70% and a resolution of Å or better. at this stage, we select five times as many negative pairs as positive pairs. we filter the combined collection of positive and negative pairs to remove redundant pairs. for each data set, we cluster the protein sequences using blastclust with a threshold of 40% sequence identity. we search for cases in which pairs of proteins link the same two clusters. if all pairs are negative, then we remove all except one randomly selected pair. if one pair is positive and others are negative, then we remove the negative pairs. if two or more pairs are positive, then we remove all the pairs except the positive one that has the minimal number of neighbors. finally, we remove all negative pairs that involve a protein shorter than amino acids and then randomly downsample the remaining negative pairs so that the ratio of negative to positive pairs is 3: this procedure generated two different data sets. the "core" data set contains proteins, with interacting pairs involving proteins and non-interacting pairs. the "small-scale" set contains proteins, with interacting pairs involving proteins and non-interacting pairs. we also build a subset of the "core", called "core-subset," that contains the same number of interacting pairs as the small-scale. note that we also generated three additional "core" and three additional "small-scale" data sets, using homology thresholds of 10-20and 10- and redundancy thresholds of 40% and 90%. the sizes and cross-validation results from those data sets are summarized in the additional file all data sets used in this study are available at http://noble.gs.washington.edu/proj/pips.  for each of the eight classifiers, we performed 5-fold cross-validation, repeated three times. during each svm training, an internal 5-fold cross-validation was performed to select two regularization parameters using grid search. we selected the regularization parameter associated with interacting pairs from {10- 10-..., 108}, and we selected the ratio c+/c- from { 100}. the parameter k of nn was selected from { 15}. in each testing phase of the cross-validation, the proportion of negative examples with respect to positive examples is equal to r = in a more realistic setting, this ratio would be much higher. therefore, we extrapolate from the measured precision p to estimate the precision of each classifier in the case where all negative examples were considered in the benchmark. if v is the number of proteins, and e is the number of interacting pairs, then the ratio of negative to positive examples is r' = / - e)/e. the estimated precision thus becomes p/). to evaluate the statistical significance of the differences in performance observed in the top row of figure we performed wilcoxon signed-rank tests on the area under the precision-recall curve across the splits of the 3×5cv of the four data sets. the bottom row of panels in figure shows the results of this test. in the graph, an edge from method a to b indicates that method a outperforms method b according to the wilcoxon test with p <. redundant edges have been removed for clarity; i.e., the figure shows the transitive reduction of the full graph. the signed-rank results show two expected trends across the three benchmarks. first, in general, structure-based methods perform better than the corresponding sequence-based methods. in particular, classifiers that use a mammoth kernel outperform the corresponding mismatch kernel method in cases out of and never perform worse. second, svm-based methods generally perform better than nearest neighbors methods, although this trend is not as consistent. the relative performance of the svm and nn classifiers seems to depend in part upon which pairwise transformation is employed. in conjunction with the mlpk transformation, the svm outperforms the nn in all six cases. in contrast, when the tppk transformation is employed, nn performs better than the svm in four cases, and ties in the remaining two. although the relative performance of the mlpk and tppk transformations is difficult to explain, one may speculate that the tppk definition results in a more natural measure of similarity between pairs than the mlpk one; hence, tppk is likely to behave better with a nn classifier. on the other hand, the mlpk transform can be justified from the point of view of metric learning when used in combination with an svm, but mlpk remains an obscure measure of similarity between pairs, which may therefore not be optimally used by a nn classifier. finally, with respect to mlpk versus tppk, the mlpk transformation performs best overall. in out of cases, the mlpk method outperforms the corresponding tppk method. the only three exceptions are nn classifiers using sequence kernels, which are relatively poorly performing methods overall. overall, our experiments suggest that, for the prediction of protein-protein interactions from structure, the mlpk svm combined with mammoth is the best method among the eight that we considered. this method performs best on all three benchmarks. having selected the best performing method, we used an svm to predict novel interactions among all pairs of proteins in a non-redundant subset of the pdb. we used the pisces database with a percent identify cutoff of 90% and a resolution of Å or better, eliminating proteins shorter than amino acids and longer than amino acids. this set contains structures. prior to the svm analysis, we eliminated pairs that are similar to any member of the training set. specifically, for a candidate pair, we define the distance to a training set pair as where e is the blast e-value assigned to query p against target q. we only consider candidate pairs for which the minimum distance to any member of the training set exceeds an e-value of. this filtering reduces the complete set from, to, pairs. we applied to each of these pairs the svm mlpk mammoth method, trained on 90% of the small-scale dip benchmark, and we used the held-out 10% of the data to convert the svm discriminants to probabilities. at a threshold of 99%, the svm predicts novel interactions; at 90%, the svm predicts pairs. the latter set is publicly available via the yeast resource center public data repository http://yeastrc.org/pdr. discussion we presented eight machine learning methods for large-scale prediction of interactions between pairs of protein structures. the methods use either a nn or a svm classifier, coupled with two kernels on protein sequences or structures, coupled with two methods for representing similarities between pairs of protein. in our cross-validation experiment, we observed--not surprisingly--that structure-based methods are generally more accurate than the methods based on sequence. among structure-based classifiers, the svm with an mlpk kernel performs best. the classifiers described here only use structural or sequence information about the proteins. further improvements in prediction accuracy could probably be obtained by taking into account other information such as expression profiles or sub-cellular localization. the integration of such heterogeneous data could for example be carried out by creating different kernels from each type of information, and merging the information by forming a linear combination of all individual kernels. one shortcoming of our approach is that each protein interaction is predicted independently. we believe that taking into account the topology of the whole interaction graph might improve the accuracy of the predictions. however, generalizing binary prediction to more complex prediction, such as graph prediction, remains a challenging issue and an active research topic in the machine learning community. in the course of this study we generated eight benchmarks that we make publicly available and which can be used to assess the performance of other methods for the automated prediction of interaction between protein structures. the benchmark generation depends on two parameters: the homology threshold, to be input to psi-blast to find a proxy pair, and the redundancy threshold to be input to blastclust. for each of the benchmarks, we set the values of the homology threshold to 10- and 10- and the values of the redundancy thresholds to and leading to four datasets for each benchmark. here, we reported the cross-validation results of only one dataset; however, the conclusions do not change when we consider the cross-validation experiments on the other benchmark data sets. finally, as pointed out in the introduction, the main motivation behind this work is to perform a large-scale screening of pairs of proteins likely to interact, before validating the prediction by more expensive computational methods such as docking, or experimental methods for pairs of particular interest. this will be the subject of future work.  the eight presented machine learning methods based on kernel methods to predict the binary interaction between protein structures were evaluated on a data set of reliable interations. we draw three main conclusions from our experiments. first, using structure we can more accurately predict interactions than using sequence alone. second, svm methods compare favorably against the methods based on the nearest neighbor classifier. third, the best choice of pairwise metric associated with the predictor is mlpk rather than tppk. authors' contributions mh, jpv and wsn designed the study and drafted the manuscript. mh performed the experiments. mr included the predicted interactions in the yrc database. mh, mr, jpv and wsn read and approved the final manuscript. supplementary material additional file 1 cross-validation results on additional core and small-scale datasets. the file contains the size and cross-validation results of eight datasets, including three additional core and three additional small-scale. the core and small-scale datasets are parametrized by two parameters during generation, the homology threshold and the redundancy threshold. the datasets correspond to two values for the homology threshold and two values of redundancy threshold. click here for file acknowledgements this work was funded in part by nih award p rr 
among structure-based classifiers, the svm algorithm, combined with the metric learning pairwise kernel and the mammoth kernel, performs best in our experiments. our approach is to discriminate between interacting and non-interacting protein pairs using a statistical pattern recognition method known as a support vector machine. approaches based on docking protein structures solve a variant of this problem, but these methods remain very computationally intensive and will not scale in the near future to the detection of interactions at the level of an interactome, involving millions of candidate pairs of proteins. while experimental methods for identifying these interactions remain costly and often noisy, the increasing quantity of solved 3d protein structures suggests that in silico methods to predict interactions between two protein structures will play an increasingly important role in screening candidate interacting pairs. the prediction of protein-protein interactions is an important step toward the elucidation of protein functions and the understanding of the molecular mechanisms inside the cell. here, we describe a computational method to predict efficiently in silico whether two protein structures interact. this yes/no question is presumably easier to answer than the standard protein docking question, "how do these two protein structures interact?" we demonstrate that our structure-based method performs well on this task and scales well to the size of an interactome.
 the prediction of protein-protein interactions is an important step toward the elucidation of protein functions and the understanding of the molecular mechanisms inside the cell. while experimental methods for identifying these interactions remain costly and often noisy, the increasing quantity of solved 3d protein structures suggests that in silico methods to predict interactions between two protein structures will play an increasingly important role in screening candidate interacting pairs. approaches using the knowledge of the structure are presumably more accurate than those based on sequence only. approaches based on docking protein structures solve a variant of this problem, but these methods remain very computationally intensive and will not scale in the near future to the detection of interactions at the level of an interactome, involving millions of candidate pairs of proteins.  here, we describe a computational method to predict efficiently in silico whether two protein structures interact. this yes/no question is presumably easier to answer than the standard protein docking question, "how do these two protein structures interact?" our approach is to discriminate between interacting and non-interacting protein pairs using a statistical pattern recognition method known as a support vector machine. we demonstrate that our structure-based method performs well on this task and scales well to the size of an interactome.  the use of structure information for the prediction of protein interaction yields significantly better performance than other sequence-based methods. among structure-based classifiers, the svm algorithm, combined with the metric learning pairwise kernel and the mammoth kernel, performs best in our experiments. 