BACKGROUND
protein-protein interactions  play critical roles in many cellular biological processes, such as signal transduction, immune response, and cellular organization.
analysis of ppi is therefore of great importance and may shed light on drug target detection and aid in therapy design  <cit> .
biochemical assays, chromatography, and similar small-scale experimental methods have long been used to identify novel ppis, but these only contribute to a low coverage of the whole ppi database due to their poor efficacies  <cit> .
high-throughput technologies, such as yeast two-hybrid screens   <cit>  and mass spectrometric protein complex identification   <cit> , have generated copious data, however, they are expensive and time consuming.
in addition, these methods may not be applicable to proteins from all organisms and often produce false-positive results  <cit> .
therefore, high-throughput computational methods are needed to identify ppis with high quality and accuracy.
recently, many computational methods have been generated to solve this problem.
of these, some have attempted to mine new protein information, whereas others involved the development of new machine-learning algorithms.
for protein information mining, shen et.al regarded any three continuous amino acids as a unit and calculated the frequencies of those conjoint triads in the protein sequences.
they demonstrated that ppis could be predicted by sequences alone  <cit> .
several other methods, such as autocovariance   <cit>  and amino acid index distribution  <cit>  were developed to extract features such as physical chemical properties, frequencies, and locations of amino acids to represent a protein sequence.
considering the high dimensions of the features, dimension reduction techniques have been used.
for machine-learning algorithms, support vector machine  and its derivatives  <cit> , random forest  <cit>  and neural networks  <cit> , have been applied.
however, most studies provided only the results of cross-validation, and did not test prediction results using external datasets  <cit> .
deep-learning algorithms, which mimic the deep neural connections and learning processes of the human brain, have received considerable attention due to their successful applications in speech and image recognition  <cit> , natural language understanding  <cit>  and decision making  <cit> .
compared to traditional machine learning methods, deep-learning algorithms can handle large-scale raw and complex data and automatically learn useful and more abstract features  <cit> .
in recent years, these algorithms have been applied to bioinformatics to manage increasing amounts and dimensions of data generated by high throughput technique .
for genome regulation function prediction, for example, xiong et al.
applied a deep neural network model to predict dna variants causing aberrant splicing.
their method was more accurate than traditional models  <cit> .
the deepbind model constructed by alipanahi and colleagues using convolutional networks could predict sequence specificities of dna- and rna-binding proteins, and identify binding motifs  <cit> .
identifying functional effects of noncoding variants is a major challenge in human genetics.
deepsea developed by zhou et al.
could directly learn a regulatory sequence code from large-scale chromatin-profiling data, enabling prediction of chromatin effects of sequence alterations with single-nucleotide sensitivity  <cit> .
after that, the dnaq model constructed by quang and coworkers achieved more than a 50% relative improvement compared to other models for predicting the function of non-coding dna  <cit> .
for protein function prediction, spencer et al.
used a deep belief network  to predict protein secondary structures and they achieved an accuracy of  <dig> %  <cit> .
sheng and colleagues improved the prediction accuracy to 84% using deep convolutional neural fields  <cit> .
heffernan et al.’s algorithm can not only predict secondary structures, but also can predict backbone angles and solvent accessible surface areas  <cit> .
a more detailed summary of the application of the deep learning algorithm in computational biology can be found in a recent review  <cit> .
in this study, we applied stacked autoencoder  to study sequence-based human ppi predictions.
models based on protein sequence autocovariance coding achieved the best results on 10-fold cross-validation  and on predicting hold-out test sets.
the best model had an average accuracy of  <dig> % for the whole training benchmark dataset.
various external test sets were constructed and predicted using our model and the prediction accuracies for these ranged from  <dig>  to  <dig> %.
in addition, we trained and tested ppi models on other species, and the results were also promising.
to our knowledge, our research is the first to use a deep-learning algorithm for sequence-based ppi prediction, and we achieved prediction performance that surpassed previous methods.
datasets
benchmark dataset
we obtained the pan’s ppi dataset from http://www.csbio.sjtu.edu.cn/bioinf/lr_ppi/data.htm  <cit> .
in this dataset, the positive samples  are from the human protein references database , with removal of duplicated interactions .
negative samples  were generated by pairing proteins found in different subcellular locations.
the protein subcellular location information was from the swiss-prot database, version  <dig> , according to the following criteria.
only human proteins were collected.
sequences annotated with ambiguous or uncertain subcellular location terms, such as “potential”, “probable”, “probably”, “maybe”, or “by similarity”, were excluded.
sequences annotated with two or more locations were excluded for lack of uniqueness.
sequences annotated with “fragment” were excluded, and sequences with fewer than  <dig> amino acid residues were removed due to the possibility that they may represent fragments.
in total,  <dig>  unique proteins from six subcellular locations  were obtained.
by randomly pairing those proteins with others found in different subcellular locations, along with the addition of negative pairs from  <cit> , a total of  <dig>  negative pairs were generated.
we removed protein pairs with unusual amino acids, such as u and x to yield  <dig>  positive samples and  <dig>  negative samples to form the benchmark dataset.
the interaction networks and the degree distributions of the positive and negative sample sets of the benchmark dataset are shown in additional file  <dig> figure s <dig> and s <dig>
we mixed the positive and negative samples in the benchmark dataset and randomly selected  <dig>  pairs  as a hold-out test set for model validation, the remainder of which formed the pre-training set .
the pre-training set was trained and tested using 10-cv, and the best models were selected to predict the hold-out test set.
to test the robustness of the model, a non-redundant test set  was formed by removing pairs in the hold-out test set with a pairwise identity ≥25% to those in the pre-training set.
after the network architecture and parameters were selected, we trained with the whole benchmark dataset to construct our final ppi prediction model and used it to predict the external test sets.
external test sets
we used the following datasets as the external test sets.
<dig> hprd dataset: the  <dig> version of the hprd dataset was downloaded and after removal of pairs common to the benchmark dataset,  <dig>  pairs were obtained.
<dig> hprd nr dataset: we removed all pairs in the  <dig> hprd dataset with a pairwise identity ≥25% to those in the benchmark dataset, after which, a total of  <dig>  pairs remained.
dip dataset: the  <dig> version released database of interacting proteins  was downloaded.
after removal of pairs shared with the benchmark dataset,  <dig>  pairs were obtained.
hippie dataset: the newly released hippie v <dig>  was downloaded.
it contains the human ppis from  <dig> large databases.
the scores of ppis which were equal or larger than  <dig>  was regarded as ‘high quality’  data by the authors, while the scores of ppis which were lower than  <dig>  was regarded as ‘low quality’  data.
after removal of pairs shared with the benchmark dataset,  <dig> of ‘high quality ’ ppis dataset and  <dig> of ‘low quality ’ ppis dataset were obtained.
inweb_inbiomap: the newly released inweb_inbiomap was downloaded.
it contains the human ppis from  <dig> large databases.
we screened out the ppis with ‘confidence score’ equal  <dig> as the ‘high quality’  data and treated the rest as the ‘low quality’ data.
after removal of pairs shared with the benchmark dataset,  <dig> of ‘high quality’ ppis dataset and  <dig> of ‘low quality’ ppis dataset were obtained.
<dig> martin dataset: this dataset was provided by pan et al.
<cit> .
note that the samples in datasets 1– <dig> were all positive and dataset  <dig> contained both positive and negative samples.
detailed information on the benchmark dataset and the external test sets appear in additional file 2: table s <dig>
datasets from other species
we also trained and tested our models using ppi samples from other species, such as escherichia coli, drosophila, and caenorhabditis elegans.
the datasets, all obtained from dip, were provided by guo et al.
and include:
e. coli-positive dataset containing  <dig>  samples.
drosophila-positive dataset containing:  <dig>  samples.
c. elegans positive dataset containing  <dig>  samples.
the negative samples from each species were also created by pairing proteins from different subcellular locations, and, in all cases, the number of negative samples was equal to the number of positive samples.
methods
stacked autoencoder
an autoencoder is an artificial neural network that applies an unsupervised learning algorithm which infers a function to construct hidden structures from unlabeled data.
specifically, it attempts to make output x^ similar to input x, which is an encoding-decoding process.
an sae consists of multiple layers of autoencoders, which are layer-wise trained in turn, and the output of the former layer is wired to inputs of the successive layer.
consider a stacked autoencoder with n layers; the encoding process of each layer is represented by:  <dig> a1=fz <dig>
<dig> zl+1=wl1al+bl <dig>
and the decoding process is its reverse order:  <dig> an+1=fzn+ <dig>
<dig> zn+l+1=wn−l,2ab+l+bn−l, <dig>
where w
, w
, b
, b
represent the weights , w
) and biases , b
), respectively, for the kth layer autoencoder, and the useful information is stored in a
.
this process may learn a good representation of the raw input after several layers, and we can then link the output to a softmax classifier to fine-tune all the previous parameters using a back-propagation algorithm with classification errors.
the structure of a stacked autoencoder is shown in fig.
<dig> fig.
<dig> the structure of a stacked autoencoder
here, we used the sae deeplearning toolbox downloaded from .
the learning rate and the momentum of the model were the same for both human and other species, while the neurons and layers were tuned and adjusted according to the training set of different species.
the detailed information of the training model can be found in additional file  <dig>
protein sequence coding
we used two methods to code the protein sequences, one is called the autocovariance method  and the other is called the conjoint triad method .
autocovariance method
the ac method, which describes how variables at different positions are correlated and interact, has been widely used for coding proteins  <cit> .
the protein sequence is transformed by the following equation:  <dig> aclag,j=1n−lag∑i=1n−lagxi,j−∑i=1nxi,j×xi+lag,j−∑i=1nxi,j
where j refers to the j-th descriptor, i is the position of the protein sequence x, ⋅ xi,j is the normalized j-th descriptor value for i-th amino acid, n is the length of the protein sequence x, and lag is the value of the lag.
in this way, proteins with variable lengths can be coded into vectors of equal length .
in this study, j is seven ; the names and exact values of these properties are shown in additional file 4: table s <dig>  guo and colleagues  <cit>  selected a value of  <dig> for the lag and we also used this value.
consequently, the vector contains  <dig> numbers .
the codes of two proteins in a pair were normalized and concatenated as the input to the models.
conjoint triad method
the ct method was first proposed by shen et al.
to represent a protein using only its sequence information  <cit> .
first, all  <dig> amino acids are clustered into seven groups according to their dipole and side chain volumes .
next, each amino acid from a protein sequence is replaced by its cluster number.
for example, the protein sequence: p=mreivhiqag
is replaced by: p= <dig>
then, a 3-amino acid window is used to slide across the whole sequence one step at a time from the n-terminus to the c-terminus.
by calculating the frequency of the combination of each three numbers:  <dig> 111=f1121=f8⋯177=f337211=f2221=f9⋯277=f338⋯711=f7721=f14⋯777=f <dig>
the protein p is represented by a vector of  <dig> numbers, all of which are zero except for f <dig> , f <dig> , f <dig> , f <dig> , f <dig> , f <dig> , f <dig> , and f <dig> .
evaluation criteria
the performance of the models was evaluated by means of the classification accuracy, specificity, sensitivity, and precision, as defined respectively by:  <dig> accuracy=tp+tntp+tn+fp+fn
<dig> specificity=tntn+fp
<dig> senisitivity=tptp+fn
<dig> precision=tptp+fp
where tp, tn, fp, and fn represents true positive, true negative, false positive, and false negative, respectively.
RESULTS
training
the pre-training dataset was trained with 10-cv, and models with the best performance were selected to predict the hold-out test set.
because the hidden layer and the neuron numbers for each layer of sae are both critical parameters, we tried different combinations; details of these combinations are shown in additional file  <dig> figures s <dig>  s <dig> and table s <dig>
interestingly, for both the ac and ct models , one hidden layer was adequate for this task.
more specifically, a one-hidden-layer model with  <dig> neuron numbers using the ac model achieved the best results .
the overall accuracies of the ct models ranged from  <dig> % to  <dig> %, with the best average accuracy,  <dig> %, achieved at  <dig> neuron numbers .
it is worth noting that a one-hidden-layer with a medium neuron numbers was sufficient to train the dataset with relatively high accuracy; more layers and neuron numbers did not improve the predictive power.
this phenomenon was also observed by zeng et al.
on predicting the protein binding motif on dna using a convolutional neural network  method  <cit> .
this might be due to the specificity of individual task and the nature of the individual dataset.table  <dig> the 10-cv training performance of the pre-training models and their prediction performances on test sets
column 2– <dig> represent the results of 10-cv with standard deviations ranged from  <dig>  to  <dig>
test set acc.
: prediction accuracy for the hold-out test set
nr-test set acc.
: prediction accuracy for the nr-test set
then ac and ct models with the best performance with 10-cv were recruited to predict the hold-out test set.
the ac model achieved an accuracy of  <dig> %, whereas the ct model  <dig> %.
we removed all pairs in the hold-out test set with ≥25% pairwise identity with those in training set  and used these to confirm the models.
the predictive abilities of both models did not decrease appreciably with the nr-test set .
so, we obtained robust performance on 10-cv training, and for predicting the hold-out and the nr-test sets.
because the ac coding method was superior to the ct coding method for this task, we used ac in the subsequent model construction.
we built our final model with the architecture and parameters of the best trained ac model trained on pre-training dataset.
this time the whole benchmark dataset was used for training with 10-cv.
we achieved a 10-cv training accuracy as depicted in table  <dig>  which is one of the best training results compared to the previous methods using the same dataset .
the dirichlet allocation -random forest  model from pan et al.
yielded the best training accuracy.
regrettably, however, most previous research did not use external test sets to further confirm predictive abilities of their methods, including pan’s.table  <dig> the 10-cv training performance of the final model
column 2– <dig> represent the training results of 10-cv with standard deviations ranged from  <dig>  to  <dig>
prediction of external test sets
our final model was used to predict the external test sets.
we used the newest version of hprd dataset  as one of the external test sets for our model.
after excluding the protein pairs that are same in the benchmark dataset, a total of  <dig>  ppi were obtained.
our model yielded a prediction accuracy of  <dig> %.
after the removal of the protein pairs with a ≥25% pairwise sequence identity to those in the benchmark dataset , the prediction accuracy was still high  .
we compared our results with guo’s work.
using the  <dig> version of hprd to test their model, which was based on ac coding and svm algorithm, guo et al.
achieved a prediction accuracy of  <dig> %  <cit> .
redundancy removal of their test sets resulted in a prediction accuracy of  <dig> %.
this demonstrated a better prediction capacity of our model.
the  <dig> version of the dip human dataset  was also tested, and this yielded a prediction accuracy of  <dig> %  for our model.
as the training accuracy of the model of pan et al.
was slightly higher than ours, we compared prediction abilities of the two models on external test sets.
we submitted the  <dig> hprd, the  <dig> hprd nr, and the dip datasets to pan’s online server , and the returned prediction accuracies on these datasets were  <dig> %,  <dig> %, and  <dig> %, respectively.
these values were lower than those obtained with our model .table  <dig> prediction performance of the final model on external datasets
hq high quality, lq low quality
recently, a large number of human ppis have been verified due to the continually development of the high-throughput technologies.
we selected two comprehensive databases that integrated most of the newly-updated ppis databases  to test our model.
the prediction accuracy of the hippie hq was  <dig> % while the prediction accuracy of the hippie lq was  <dig> %.
the prediction accuracy of the inweb_inbiomap hq was  <dig> % while the prediction accuracy of the inweb_inbiomap lq was  <dig> %.
we noticed that our model had better prediction on the hq dataset than the lq dataset.
we also submitted the hippie hq dataset to pan’s server, and the returned prediction accuracy was  <dig> %, which was lower than that of our model .
overall, these data suggest that our model, based on sae, is a powerful and promising tool for the prediction of ppi, especially for the newly released ppis from the two comprehensive datasets.
a previously generated dataset with  <dig> positive and  <dig> negative samples   <cit>  has been utilized in a number of previous studies .
we noticed, however, that most of the previous models used this dataset for training and did not use it for testing.
as this dataset is small and has a low coverage on ppi space, the training performance of the previous research using it seems unsatisfying.
notably, zhang, et al.
only used positive samples of the  <dig> martin dataset to test their model and achieved an accuracy of  <dig> %  <cit> .
we also tested the  <dig> martin dataset with our model, and we achieved an accuracy of only about 50%, suggesting that the model nearly lost predictive ability .
we then tested the positive and negative samples separately and found the prediction accuracy for the positive samples was as high as  <dig> % , whereas for the negative samples, the prediction accuracy was only  <dig> %.
we also used pan’s web server to test positive and negative samples from the  <dig> martin dataset, and found that the prediction accuracies were nearly the same as ours .
thus, the model regarded most of the negative samples as positive.
we compared the sequence similarities of the positive and negative samples of the  <dig> martin dataset between the positive and negative samples of the benchmark dataset, respectively, and found the unsatisfied result might be due to that the negative samples of  <dig> martin dataset was much similar to the positive samples of the benchmark dataset rather than similar to the negative samples of benchmark dataset .
performance on prediction ppi from other species
we also tested the performance of our algorithm with regard to ppis from e. coli, drosophila, and c. elegans, with the same training and test data provided by guo et al.
they built their models using svm with protein coded by ac  <cit> .
here, we used 5-cv in training which could directly compare with guo’s result.
for e. coli, the model was  <dig> layers and for each layer  <dig>   <dig>  and  <dig> neurons were used , and this achieved an average training accuracy of  <dig> %.
for drosophila, the model structure had three layers , and it achieved an average training accuracy of  <dig> %.
for c. elegans, the model structure  <cit>  achieved an average training accuracy of  <dig> %.
the detailed training results of our models with guo et al.
’s training accuracies as comparison are listed in table  <dig>  it can be seen that for c. elegans, we achieved comparable accuracy to guo et al.’s model, while for e. coli and drosophila, our accuracies were higher.
overall, these results demonstrate the power of our algorithm for different species.table  <dig> training performance on ppis from other species
e. coli
drosophila
c. elegans
colum 2– <dig> are the training results of 5-cv with standard deviations ranged from  <dig>  to  <dig>
discussion
deep-learning algorithms have been used in many fields and their applications in bioinformatics are increasing.
however, these powerful methods have not yet been extended to the study of ppi.
thus, in this study, we used a deep-learning algorithm-sae, in combination with two widely-used protein sequence coding methods ac and ct, to study human ppis.
the performance of our model suggests that the sae algorithm is robust, and that the ac coding method is superior to ct coding for this task.
the training accuracy of our model on the benchmark dataset was comparable to, or higher than, previous models.
our model also had good predictive ability for other external test sets, which were not tested in most previous studies.
it is noteworthy that our model gave a satisfying prediction accuracy for a large number of newly verified ppis.
although pan et al.’s model achieved the highest training accuracy , our prediction accuracies for the three external test sets were significantly better.
in addition, we applied our algorithm to train and test ppis from other species, and performance was promising.
proteins interact with one another through a group of amino acids or domains, so the success of our sae algorithm may be due to its powerful generalization capacity on protein sequence input codons to learn hidden interaction features.
although many previous models performed considerably worse on the  <dig> martin dataset, sufficient evidence was not available to explain why this happened.
by testing positive and negative samples separately and analyzing sequence similarities between the test and training sets, we found less sequence similarity between the martin  <dig> negative samples and the training negative samples, and we believed that this contributed to unsatisfying prediction accuracy.
because the data were based on unbalanced positive and negative samples, likely the algorithm did not learn many more features than sequence similarity to discriminate between positive and negative datasets .
considering that only ~ <dig>  proteins with verified subcellular location were available to construct the negative samples , the combined number of protein pairs was insufficient to cover the negative ppi space, prohibiting construction of a reliable ppi prediction model, something also mentioned in pan et al.’s paper  <cit> .
our analysis re-emphasizes the need to construct a solid negative dataset with wide coverage of proteins for ppi prediction, in addition to expanding the absolute number of ppi samples for training.
this idea agrees with the concept of big data, which emphasizes data complexity besides of data volume.
some consideration has been made for selection of negative samples.
for instance, yu and co-workers proposed that the negative and positive training sets should be balanced to achieve a high-confidence result  <cit> , but park et al.
disagreed, arguing that yu et al.
confused different purposes for ppi analysis  <cit> .
other groups have tried different methods to build negative data, but did not achieve promising results  <cit> .
we suggest that future work should focus on the construction of a solid and reasonable negative training set, covering negative ppi space as much as possible, to improve the overall prediction accuracy for external datasets.
for protein sequence coding, we used the pre-defined feature extraction methods of ac and ct and the model performed well for predicting ppis.
either ac or ct has undergone predefined feature selection.
with ac coding, physical chemical properties were selected by human knowledge, whereas with ct coding, amino acid classification was made manually.
using pre-defined features for protein function prediction with deep learning algorithm has been common in previous work .
this deviated somewhat from the essence of deep learning: automatic feature extraction.
future work may focus on developing novel methods for best representing raw protein sequence information.
CONCLUSIONS
in this study, we applied the deep-learning algorithm, sae, for sequence-based ppi prediction.
the best model achieved an average training accuracy of  <dig> % on 10-cv training.
its predictive accuracies for diverse external datasets ranged from  <dig> % to  <dig> %.
furthermore, we trained the datasets from other species, such as e. coli, drosophila, and c. elegans, and results were also promising.
to our knowledge, this research is the first to apply a deep-learning algorithm to sequence-based ppi prediction, and the results demonstrate its potential in this field.
additional files
additional file 1: detailed description of the benchmark dataset.
figure s <dig>   protein interaction network of the positive samples from the benchmark dataset and  negative pairs’ network from the benchmark dataset.
figure s <dig>  degree distribution of the protein interaction network;  positive samples from the benchmark dataset, and  negative samples from the benchmark dataset.
additional file 2: detailed information for the benchmark and the external test sets.
table s <dig>  detailed information for the benchmark and the external test sets.
additional file 3: detailed information about the training model.
additional file 4: detailed information about protein coding methods.
table s <dig>  classification of amino acids of ct coding method.
table s <dig>  physicochemical properties of amino acid for calculating ac.
additional file 5: detailed description of the parameter selection.
figure s <dig>  the 10-cv training accuracies of the pre-training model in response to increasing numbers of neurons in the one-layer model:  ac coding model  and  ct coding model .
figure s <dig>  the 10-cv training accuracies of the pre-training models in response to increasing numbers of neurons in the two-layer models:  ac model and  ct model.
table s <dig>  the 10-cv training accuracies of the three-layer models.
additional file 6: more details about the redundancy removal of the test set.
additional file 7: the ppi pairs in  <dig> version of dip dataset.
additional file 8: detailed analysis of  <dig> martin dataset.
table s <dig>  the predictive performance on the  <dig> martin dataset.
table s <dig>  sequence similarities between the  <dig> martin dataset and the benchmark dataset.
additional file 9: detailed analysis of the prediction accuracies of the test sets.
table s <dig>  percent of proteins in the test sets having ≥30% sequence identity to those in pre-training/whole benchmark dataset and the prediction accuracy.
figure s <dig>  relationship between the prediction accuracy and the percent of proteins in a test set with ≥30% sequence identity to those in the training set.
abbreviations
10-cv10-fold cross-validation
acautocovariance
cnnconvolutional neural network
ctconjoint triad method
dbndeep belief network
fnfalse negative
fpfalse positive
ldathe dirichlet allocation
ppisprotein protein interactions
rfrandom forest
saestacked autoencoder
svmsupport vector machine
tntrue negative
tptrue positive
electronic supplementary material
the online version of this article  contains supplementary material, which is available to authorized users.
we thank youjun xu, ziwei dai, shuaishi gao, and other members of the lai laboratory for insightful and helpful suggestions.
funding
this work was supported in part by the national natural science foundation of china  and the ministry of science and technology of china .
availability of data and materials
the web server is repharma.pku.edu.cn/ppi.
the benchmark datasets and external test datasets can be downloaded according to the references mentioned in the main text.
authors’ contributions
pjf and llh conceived the study; stl performed the data collection, training, prediction and analysis; stl, pjf and llh wrote the paper; zb constructed the server; all authors contributed to the revised and approved the final manuscript.
competing interests
the authors declare that they have no competing interests.
consent for publication
not applicable.
ethics approval and consent to participate
not applicable.
publisher’s note
springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

 protein-protein interactions play critical roles in many cellular biological processes, such as signal transduction, immune response, and cellular organization. analysis of ppi is therefore of great importance and may shed light on drug target detection and aid in therapy design. biochemical assays, chromatography, and similar small-scale experimental methods have long been used to identify novel ppis, but these only contribute to a low coverage of the whole ppi database due to their poor efficacies. high-throughput technologies, such as yeast two-hybrid screens and mass spectrometric protein complex identification, have generated copious data, however, they are expensive and time consuming. in addition, these methods may not be applicable to proteins from all organisms and often produce false-positive results. therefore, high-throughput computational methods are needed to identify ppis with high quality and accuracy. recently, many computational methods have been generated to solve this problem. of these, some have attempted to mine new protein information, whereas others involved the development of new machine-learning algorithms. for protein information mining, shen et.al regarded any three continuous amino acids as a unit and calculated the frequencies of those conjoint triads in the protein sequences. they demonstrated that ppis could be predicted by sequences alone. several other methods, such as autocovariance and amino acid index distribution were developed to extract features such as physical chemical properties, frequencies, and locations of amino acids to represent a protein sequence. considering the high dimensions of the features, dimension reduction techniques have been used. for machine-learning algorithms, support vector machine and its derivatives, random forest and neural networks, have been applied. however, most studies provided only the results of cross-validation, and did not test prediction results using external datasets. deep-learning algorithms, which mimic the deep neural connections and learning processes of the human brain, have received considerable attention due to their successful applications in speech and image recognition, natural language understanding and decision making. compared to traditional machine learning methods, deep-learning algorithms can handle large-scale raw and complex data and automatically learn useful and more abstract features. in recent years, these algorithms have been applied to bioinformatics to manage increasing amounts and dimensions of data generated by high throughput technique. for genome regulation function prediction, for example, xiong et al. applied a deep neural network model to predict dna variants causing aberrant splicing. their method was more accurate than traditional models. the deepbind model constructed by alipanahi and colleagues using convolutional networks could predict sequence specificities of dna- and rna-binding proteins, and identify binding motifs. identifying functional effects of noncoding variants is a major challenge in human genetics. deepsea developed by zhou et al. could directly learn a regulatory sequence code from large-scale chromatin-profiling data, enabling prediction of chromatin effects of sequence alterations with single-nucleotide sensitivity. after that, the dnaq model constructed by quang and coworkers achieved more than a 50% relative improvement compared to other models for predicting the function of non-coding dna. for protein function prediction, spencer et al. used a deep belief network to predict protein secondary structures and they achieved an accuracy of %. sheng and colleagues improved the prediction accuracy to 84% using deep convolutional neural fields. heffernan et al.’s algorithm can not only predict secondary structures, but also can predict backbone angles and solvent accessible surface areas. a more detailed summary of the application of the deep learning algorithm in computational biology can be found in a recent review. in this study, we applied stacked autoencoder to study sequence-based human ppi predictions. models based on protein sequence autocovariance coding achieved the best results on 10-fold cross-validation and on predicting hold-out test sets. the best model had an average accuracy of % for the whole training benchmark dataset. various external test sets were constructed and predicted using our model and the prediction accuracies for these ranged from to %. in addition, we trained and tested ppi models on other species, and the results were also promising. to our knowledge, our research is the first to use a deep-learning algorithm for sequence-based ppi prediction, and we achieved prediction performance that surpassed previous methods. datasets benchmark dataset we obtained the pan’s ppi dataset from http://www.csbio.sjtu.edu.cn/bioinf/lr_ppi/data.htm. in this dataset, the positive samples are from the human protein references database, with removal of duplicated interactions. negative samples were generated by pairing proteins found in different subcellular locations. the protein subcellular location information was from the swiss-prot database, version, according to the following criteria. only human proteins were collected. sequences annotated with ambiguous or uncertain subcellular location terms, such as “potential”, “probable”, “probably”, “maybe”, or “by similarity”, were excluded. sequences annotated with two or more locations were excluded for lack of uniqueness. sequences annotated with “fragment” were excluded, and sequences with fewer than amino acid residues were removed due to the possibility that they may represent fragments. in total, unique proteins from six subcellular locations were obtained. by randomly pairing those proteins with others found in different subcellular locations, along with the addition of negative pairs from, a total of negative pairs were generated. we removed protein pairs with unusual amino acids, such as u and x to yield positive samples and negative samples to form the benchmark dataset. the interaction networks and the degree distributions of the positive and negative sample sets of the benchmark dataset are shown in additional file figure s and s we mixed the positive and negative samples in the benchmark dataset and randomly selected pairs as a hold-out test set for model validation, the remainder of which formed the pre-training set. the pre-training set was trained and tested using 10-cv, and the best models were selected to predict the hold-out test set. to test the robustness of the model, a non-redundant test set was formed by removing pairs in the hold-out test set with a pairwise identity ≥25% to those in the pre-training set. after the network architecture and parameters were selected, we trained with the whole benchmark dataset to construct our final ppi prediction model and used it to predict the external test sets. external test sets we used the following datasets as the external test sets. hprd dataset: the version of the hprd dataset was downloaded and after removal of pairs common to the benchmark dataset, pairs were obtained. hprd nr dataset: we removed all pairs in the hprd dataset with a pairwise identity ≥25% to those in the benchmark dataset, after which, a total of pairs remained. dip dataset: the version released database of interacting proteins was downloaded. after removal of pairs shared with the benchmark dataset, pairs were obtained. hippie dataset: the newly released hippie v was downloaded. it contains the human ppis from large databases. the scores of ppis which were equal or larger than was regarded as ‘high quality’ data by the authors, while the scores of ppis which were lower than was regarded as ‘low quality’ data. after removal of pairs shared with the benchmark dataset, of ‘high quality ’ ppis dataset and of ‘low quality ’ ppis dataset were obtained. inweb_inbiomap: the newly released inweb_inbiomap was downloaded. it contains the human ppis from large databases. we screened out the ppis with ‘confidence score’ equal as the ‘high quality’ data and treated the rest as the ‘low quality’ data. after removal of pairs shared with the benchmark dataset, of ‘high quality’ ppis dataset and of ‘low quality’ ppis dataset were obtained. martin dataset: this dataset was provided by pan et al.. note that the samples in datasets 1– were all positive and dataset contained both positive and negative samples. detailed information on the benchmark dataset and the external test sets appear in additional file 2: table s datasets from other species we also trained and tested our models using ppi samples from other species, such as escherichia coli, drosophila, and caenorhabditis elegans. the datasets, all obtained from dip, were provided by guo et al. and include: e. coli-positive dataset containing samples. drosophila-positive dataset containing: samples. c. elegans positive dataset containing samples. the negative samples from each species were also created by pairing proteins from different subcellular locations, and, in all cases, the number of negative samples was equal to the number of positive samples. methods stacked autoencoder an autoencoder is an artificial neural network that applies an unsupervised learning algorithm which infers a function to construct hidden structures from unlabeled data. specifically, it attempts to make output x^ similar to input x, which is an encoding-decoding process. an sae consists of multiple layers of autoencoders, which are layer-wise trained in turn, and the output of the former layer is wired to inputs of the successive layer. consider a stacked autoencoder with n layers; the encoding process of each layer is represented by: a1=fz zl+1=wl1al+bl and the decoding process is its reverse order: an+1=fzn+ zn+l+1=wn−l,2ab+l+bn−l, where w, w, b, b represent the weights, w ) and biases, b ), respectively, for the kth layer autoencoder, and the useful information is stored in a. this process may learn a good representation of the raw input after several layers, and we can then link the output to a softmax classifier to fine-tune all the previous parameters using a back-propagation algorithm with classification errors. the structure of a stacked autoencoder is shown in fig.  fig. the structure of a stacked autoencoder here, we used the sae deeplearning toolbox downloaded from. the learning rate and the momentum of the model were the same for both human and other species, while the neurons and layers were tuned and adjusted according to the training set of different species. the detailed information of the training model can be found in additional file protein sequence coding we used two methods to code the protein sequences, one is called the autocovariance method and the other is called the conjoint triad method. autocovariance method the ac method, which describes how variables at different positions are correlated and interact, has been widely used for coding proteins. the protein sequence is transformed by the following equation: aclag,j=1n−lag∑i=1n−lagxi,j−∑i=1nxi,j×xi+lag,j−∑i=1nxi,j where j refers to the j-th descriptor, i is the position of the protein sequence x, ⋅ xi,j is the normalized j-th descriptor value for i-th amino acid, n is the length of the protein sequence x, and lag is the value of the lag. in this way, proteins with variable lengths can be coded into vectors of equal length. in this study, j is seven; the names and exact values of these properties are shown in additional file 4: table s guo and colleagues selected a value of for the lag and we also used this value. consequently, the vector contains numbers. the codes of two proteins in a pair were normalized and concatenated as the input to the models. conjoint triad method the ct method was first proposed by shen et al. to represent a protein using only its sequence information. first, all amino acids are clustered into seven groups according to their dipole and side chain volumes. next, each amino acid from a protein sequence is replaced by its cluster number. for example, the protein sequence: p=mreivhiqag is replaced by: p= then, a 3-amino acid window is used to slide across the whole sequence one step at a time from the n-terminus to the c-terminus. by calculating the frequency of the combination of each three numbers: 111=f1121=f8⋯177=f337211=f2221=f9⋯277=f338⋯711=f7721=f14⋯777=f the protein p is represented by a vector of numbers, all of which are zero except for f, f, f, f, f, f, f, and f. evaluation criteria the performance of the models was evaluated by means of the classification accuracy, specificity, sensitivity, and precision, as defined respectively by: accuracy=tp+tntp+tn+fp+fn specificity=tntn+fp senisitivity=tptp+fn precision=tptp+fp where tp, tn, fp, and fn represents true positive, true negative, false positive, and false negative, respectively.  training the pre-training dataset was trained with 10-cv, and models with the best performance were selected to predict the hold-out test set. because the hidden layer and the neuron numbers for each layer of sae are both critical parameters, we tried different combinations; details of these combinations are shown in additional file figures s s and table s interestingly, for both the ac and ct models, one hidden layer was adequate for this task. more specifically, a one-hidden-layer model with neuron numbers using the ac model achieved the best results. the overall accuracies of the ct models ranged from % to %, with the best average accuracy, %, achieved at neuron numbers. it is worth noting that a one-hidden-layer with a medium neuron numbers was sufficient to train the dataset with relatively high accuracy; more layers and neuron numbers did not improve the predictive power. this phenomenon was also observed by zeng et al. on predicting the protein binding motif on dna using a convolutional neural network method. this might be due to the specificity of individual task and the nature of the individual dataset.table the 10-cv training performance of the pre-training models and their prediction performances on test sets column 2– represent the results of 10-cv with standard deviations ranged from to test set acc.: prediction accuracy for the hold-out test set nr-test set acc.: prediction accuracy for the nr-test set then ac and ct models with the best performance with 10-cv were recruited to predict the hold-out test set. the ac model achieved an accuracy of %, whereas the ct model %. we removed all pairs in the hold-out test set with ≥25% pairwise identity with those in training set and used these to confirm the models. the predictive abilities of both models did not decrease appreciably with the nr-test set. so, we obtained robust performance on 10-cv training, and for predicting the hold-out and the nr-test sets. because the ac coding method was superior to the ct coding method for this task, we used ac in the subsequent model construction. we built our final model with the architecture and parameters of the best trained ac model trained on pre-training dataset. this time the whole benchmark dataset was used for training with 10-cv. we achieved a 10-cv training accuracy as depicted in table  which is one of the best training results compared to the previous methods using the same dataset. the dirichlet allocation -random forest model from pan et al. yielded the best training accuracy. regrettably, however, most previous research did not use external test sets to further confirm predictive abilities of their methods, including pan’s.table the 10-cv training performance of the final model column 2– represent the training results of 10-cv with standard deviations ranged from to prediction of external test sets our final model was used to predict the external test sets. we used the newest version of hprd dataset as one of the external test sets for our model. after excluding the protein pairs that are same in the benchmark dataset, a total of ppi were obtained. our model yielded a prediction accuracy of %. after the removal of the protein pairs with a ≥25% pairwise sequence identity to those in the benchmark dataset, the prediction accuracy was still high. we compared our results with guo’s work. using the version of hprd to test their model, which was based on ac coding and svm algorithm, guo et al. achieved a prediction accuracy of %. redundancy removal of their test sets resulted in a prediction accuracy of %. this demonstrated a better prediction capacity of our model. the version of the dip human dataset was also tested, and this yielded a prediction accuracy of % for our model. as the training accuracy of the model of pan et al. was slightly higher than ours, we compared prediction abilities of the two models on external test sets. we submitted the hprd, the hprd nr, and the dip datasets to pan’s online server, and the returned prediction accuracies on these datasets were %, %, and %, respectively. these values were lower than those obtained with our model.table prediction performance of the final model on external datasets hq high quality, lq low quality recently, a large number of human ppis have been verified due to the continually development of the high-throughput technologies. we selected two comprehensive databases that integrated most of the newly-updated ppis databases to test our model. the prediction accuracy of the hippie hq was % while the prediction accuracy of the hippie lq was %. the prediction accuracy of the inweb_inbiomap hq was % while the prediction accuracy of the inweb_inbiomap lq was %. we noticed that our model had better prediction on the hq dataset than the lq dataset. we also submitted the hippie hq dataset to pan’s server, and the returned prediction accuracy was %, which was lower than that of our model. overall, these data suggest that our model, based on sae, is a powerful and promising tool for the prediction of ppi, especially for the newly released ppis from the two comprehensive datasets. a previously generated dataset with positive and negative samples has been utilized in a number of previous studies. we noticed, however, that most of the previous models used this dataset for training and did not use it for testing. as this dataset is small and has a low coverage on ppi space, the training performance of the previous research using it seems unsatisfying. notably, zhang, et al. only used positive samples of the martin dataset to test their model and achieved an accuracy of %. we also tested the martin dataset with our model, and we achieved an accuracy of only about 50%, suggesting that the model nearly lost predictive ability. we then tested the positive and negative samples separately and found the prediction accuracy for the positive samples was as high as %, whereas for the negative samples, the prediction accuracy was only %. we also used pan’s web server to test positive and negative samples from the martin dataset, and found that the prediction accuracies were nearly the same as ours. thus, the model regarded most of the negative samples as positive. we compared the sequence similarities of the positive and negative samples of the martin dataset between the positive and negative samples of the benchmark dataset, respectively, and found the unsatisfied result might be due to that the negative samples of martin dataset was much similar to the positive samples of the benchmark dataset rather than similar to the negative samples of benchmark dataset. performance on prediction ppi from other species we also tested the performance of our algorithm with regard to ppis from e. coli, drosophila, and c. elegans, with the same training and test data provided by guo et al. they built their models using svm with protein coded by ac. here, we used 5-cv in training which could directly compare with guo’s result. for e. coli, the model was layers and for each layer and neurons were used, and this achieved an average training accuracy of %. for drosophila, the model structure had three layers, and it achieved an average training accuracy of %. for c. elegans, the model structure achieved an average training accuracy of %. the detailed training results of our models with guo et al. ’s training accuracies as comparison are listed in table  it can be seen that for c. elegans, we achieved comparable accuracy to guo et al.’s model, while for e. coli and drosophila, our accuracies were higher. overall, these results demonstrate the power of our algorithm for different species.table training performance on ppis from other species e. coli drosophila c. elegans colum 2– are the training results of 5-cv with standard deviations ranged from to discussion deep-learning algorithms have been used in many fields and their applications in bioinformatics are increasing. however, these powerful methods have not yet been extended to the study of ppi. thus, in this study, we used a deep-learning algorithm-sae, in combination with two widely-used protein sequence coding methods ac and ct, to study human ppis. the performance of our model suggests that the sae algorithm is robust, and that the ac coding method is superior to ct coding for this task. the training accuracy of our model on the benchmark dataset was comparable to, or higher than, previous models. our model also had good predictive ability for other external test sets, which were not tested in most previous studies. it is noteworthy that our model gave a satisfying prediction accuracy for a large number of newly verified ppis. although pan et al.’s model achieved the highest training accuracy, our prediction accuracies for the three external test sets were significantly better. in addition, we applied our algorithm to train and test ppis from other species, and performance was promising. proteins interact with one another through a group of amino acids or domains, so the success of our sae algorithm may be due to its powerful generalization capacity on protein sequence input codons to learn hidden interaction features. although many previous models performed considerably worse on the martin dataset, sufficient evidence was not available to explain why this happened. by testing positive and negative samples separately and analyzing sequence similarities between the test and training sets, we found less sequence similarity between the martin negative samples and the training negative samples, and we believed that this contributed to unsatisfying prediction accuracy. because the data were based on unbalanced positive and negative samples, likely the algorithm did not learn many more features than sequence similarity to discriminate between positive and negative datasets. considering that only ~ proteins with verified subcellular location were available to construct the negative samples, the combined number of protein pairs was insufficient to cover the negative ppi space, prohibiting construction of a reliable ppi prediction model, something also mentioned in pan et al.’s paper. our analysis re-emphasizes the need to construct a solid negative dataset with wide coverage of proteins for ppi prediction, in addition to expanding the absolute number of ppi samples for training. this idea agrees with the concept of big data, which emphasizes data complexity besides of data volume. some consideration has been made for selection of negative samples. for instance, yu and co-workers proposed that the negative and positive training sets should be balanced to achieve a high-confidence result, but park et al. disagreed, arguing that yu et al. confused different purposes for ppi analysis. other groups have tried different methods to build negative data, but did not achieve promising results. we suggest that future work should focus on the construction of a solid and reasonable negative training set, covering negative ppi space as much as possible, to improve the overall prediction accuracy for external datasets. for protein sequence coding, we used the pre-defined feature extraction methods of ac and ct and the model performed well for predicting ppis. either ac or ct has undergone predefined feature selection. with ac coding, physical chemical properties were selected by human knowledge, whereas with ct coding, amino acid classification was made manually. using pre-defined features for protein function prediction with deep learning algorithm has been common in previous work. this deviated somewhat from the essence of deep learning: automatic feature extraction. future work may focus on developing novel methods for best representing raw protein sequence information.  in this study, we applied the deep-learning algorithm, sae, for sequence-based ppi prediction. the best model achieved an average training accuracy of % on 10-cv training. its predictive accuracies for diverse external datasets ranged from % to %. furthermore, we trained the datasets from other species, such as e. coli, drosophila, and c. elegans, and results were also promising. to our knowledge, this research is the first to apply a deep-learning algorithm to sequence-based ppi prediction, and the results demonstrate its potential in this field. additional files additional file 1: detailed description of the benchmark dataset. figure s protein interaction network of the positive samples from the benchmark dataset and negative pairs’ network from the benchmark dataset. figure s degree distribution of the protein interaction network; positive samples from the benchmark dataset, and negative samples from the benchmark dataset. additional file 2: detailed information for the benchmark and the external test sets. table s detailed information for the benchmark and the external test sets. additional file 3: detailed information about the training model. additional file 4: detailed information about protein coding methods. table s classification of amino acids of ct coding method. table s physicochemical properties of amino acid for calculating ac. additional file 5: detailed description of the parameter selection. figure s the 10-cv training accuracies of the pre-training model in response to increasing numbers of neurons in the one-layer model: ac coding model and ct coding model. figure s the 10-cv training accuracies of the pre-training models in response to increasing numbers of neurons in the two-layer models: ac model and ct model. table s the 10-cv training accuracies of the three-layer models. additional file 6: more details about the redundancy removal of the test set. additional file 7: the ppi pairs in version of dip dataset. additional file 8: detailed analysis of martin dataset. table s the predictive performance on the martin dataset. table s sequence similarities between the martin dataset and the benchmark dataset. additional file 9: detailed analysis of the prediction accuracies of the test sets. table s percent of proteins in the test sets having ≥30% sequence identity to those in pre-training/whole benchmark dataset and the prediction accuracy. figure s relationship between the prediction accuracy and the percent of proteins in a test set with ≥30% sequence identity to those in the training set. abbreviations 10-cv10-fold cross-validation acautocovariance cnnconvolutional neural network ctconjoint triad method dbndeep belief network fnfalse negative fpfalse positive ldathe dirichlet allocation ppisprotein protein interactions rfrandom forest saestacked autoencoder svmsupport vector machine tntrue negative tptrue positive electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. we thank youjun xu, ziwei dai, shuaishi gao, and other members of the lai laboratory for insightful and helpful suggestions. funding this work was supported in part by the national natural science foundation of china and the ministry of science and technology of china. availability of data and materials the web server is repharma.pku.edu.cn/ppi. the benchmark datasets and external test datasets can be downloaded according to the references mentioned in the main text. authors’ contributions pjf and llh conceived the study; stl performed the data collection, training, prediction and analysis; stl, pjf and llh wrote the paper; zb constructed the server; all authors contributed to the revised and approved the final manuscript. competing interests the authors declare that they have no competing interests. consent for publication not applicable. ethics approval and consent to participate not applicable. publisher’s note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
though various computational methods for predicting ppi have been developed, their robustness for prediction with external datasets is unknown. the prediction accuracies for various external datasets ranged from % to %, which are superior to those achieved with previous methods. deep-learning algorithms have achieved successful results in diverse areas, but their effectiveness for ppi prediction has not been tested. we used a stacked autoencoder, a type of deep-learning algorithm, to study the sequence-based ppi prediction. to our knowledge, this research is the first to apply a deep-learning algorithm to sequence-based ppi prediction, and the results demonstrate its potential in this field. protein-protein interactions are critical for many biological processes. the best model achieved an average accuracy of % with 10-fold cross-validation. the online version of this article contains supplementary material, which is available to authorized users.
 protein-protein interactions are critical for many biological processes. it is therefore important to develop accurate high-throughput methods for identifying ppi to better understand protein function, disease occurrence, and therapy design. though various computational methods for predicting ppi have been developed, their robustness for prediction with external datasets is unknown. deep-learning algorithms have achieved successful results in diverse areas, but their effectiveness for ppi prediction has not been tested.  we used a stacked autoencoder, a type of deep-learning algorithm, to study the sequence-based ppi prediction. the best model achieved an average accuracy of % with 10-fold cross-validation. the prediction accuracies for various external datasets ranged from % to %, which are superior to those achieved with previous methods.  to our knowledge, this research is the first to apply a deep-learning algorithm to sequence-based ppi prediction, and the results demonstrate its potential in this field. electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. keywords deep learningprotein-protein interactionhttp://dx.doi.org/ /501100002855ministry of science and technology of the people's republic of china2016yfa0502303lai luhua http://dx.doi.org/ /501100001809national natural science foundation of china2167301081273436pei jianfeng issue-copyright-statement© the author 2017 