BACKGROUND
the intrinsically disordered proteins , also referred to as natively unfolded or intrinsically unstructured proteins, lack stable tertiary structure in vitro.
these proteins are implicated in numerous processes including cellular signal transduction, transcriptional regulation, and translation  <cit> , and their prevalence was demonstrated in several human diseases  <cit> , including cancer  <cit> , cardiovascular disease  <cit> , neurodegenerative diseases  <cit> , genetic diseases  <cit> , and amyloidoses  <cit> .
at the same time, the annotations of the idps are accumulated at a relatively low pace when compared with the growth of the number of known, non-redundant protein sequences.
over the last decade numerous sequence-derived characteristics, including low complexity  <cit> , which was proposed in  <cit> , high net charge and low content of hydrophobic amino acids  <cit> , lack of regular secondary structure  <cit> , to name just a few, were found to differentiate between disordered and ordered regions.
the abovementioned results suggest that disorder can be predicted from the sequence and they motivate the development of computational models for the prediction of the disordered regions.
several such predictors were already developed; see  <cit>  for a recent review.
majority of the existing predictors generate the disorder predictions for each residue in the input protein chain.
these per-residue predictors can be divided into  <dig> categories: i) methods that utilize the relative propensity of amino acids to form disorder/ordered regions which include globplot  <cit> , iupred  <cit> , foldindex  <cit> , and ucon  <cit> ; ii) methods that are based on classifiers generated with the help of machine learning algorithms, such as dispro  <cit> , disopred  <cit> , disopred <dig>  <cit> , prdos  <cit> , poodle predictors  <cit> , pondr predictors  <cit> , spritz  <cit> , profbval  <cit> , dispssmp  <cit> , dispssmp <dig>  <cit> , iup  <cit> , norsnet  <cit>  and ond-crfs  <cit> ; iii) meta-approach methods that are based on a consensus of multiple base predictors including multicom   <cit> , metaprdos  <cit>  and recent md  <cit> , mfdp  <cit> , and pondr-fit  <cit>  predictors; and iv) approaches that find disordered residues through an analysis of the predicted 3d structural models such as prdos  <cit>  and disoclust  <cit> .
there are also methods that predict the propensity of the entire protein chain to be unstructured  <cit> .
one of these approaches is based on the charge-hydropathy plots  <cit>  and another utilizes distributions of the predicted per-residue disorder scores  <cit> .
the abovementioned per-residue and per-chain methods perform the predictions in a high-throughput manner and consequently they can be used as a possible solution to close the annotation gap.
although the per-residue methods are successful in the disorder prediction at the residue level, i.e., they achieve auc  of about  <dig>   <cit>  and mcc  of about  <dig>   <cit>  when tested on large benchmark datasets, we observe that they typically make relatively substantial mistakes at the sequence-level.
more specifically, these methods may over-or under-predict the overall amount of disorder in the sequence.
tests of  <dig> recent disorder predictors that include methods from all four groups on a benchmark dataset of  <dig> chains, see table  <dig>  show that the mean average  squared errors between the native and the predicted amount of disorder vary between  <dig>  and  <dig>  .
one of the potential reasons for these errors is the fact that virtually all of the most accurate recent predictors, such as norsnet, disopred <dig>  md, pondr-fit, and mfdp, use a local sequence window to predict the disorder while the information encoded in the entire chain may reveal an overall sequence-level disorder bias.
the disorder predictors use identical  cut-off values to annotate disordered residues based on the predicted real-value propensities, and we show that these annotations can be improved if the cut-off is adjusted to match the native amount of the disorder in the entire chain, which suggests that the knowledge of this sequence bias could be useful.
we report the mse, mae, and pcc values, the percentage of chains that are over-predicted  and under-predicted, and the mae value for the over-and under-predicted chains.
the methods are sorted in the descending order by the mse values and the best values are shown in bold font.
results of the tests of significance of the differences between discon and the other methods are given in the "stat.
signif."
columns.
the tests compare the absolute and the squared errors per-chain over all  <dig> chains in our test dataset, and pearson correlation computed for  <dig> randomly selected sets of  <dig> proteins from the test dataset.
the ++ and + denote that discon is statistically significantly better with p <  <dig>  and p <  <dig> , respectively, and = denotes that the results are not significantly different.
we also report their area under curve , accuracy  and mcc for the per-residue disorder predictions generated by the ten considered predictors.
the overall disorder content was used in the past to estimate the abundance of intrinsic disorder in several protein databases  <cit> , in various protein families and classes  <cit> , and in complete proteomes  <cit> .
the high values of the disorder content were reported for several disease-related proteins  <cit> .
the content was also used for the analysis of intrinsic disorder-related protein functions  <cit> .
importantly, in all these and similar cases, the disorder content was evaluated based on the results of either binary classifiers or was derived from the per-residue disorder predictions.
as mentioned above, these per-residue disorder prediction methods may over-or under-predict the overall amount of disorder in the sequence.
this observation and the fact that the knowledge of the disorder content in a given protein or in a set of proteins of interest or in an entire proteome can be utilized to investigate numerous important hypotheses motivate the development of new computational tools for the accurate prediction of the disorder content.
we propose a novel method, named discon , that aims to provide accurate sequence-based predictions of the disorder content.
our approach is based on the premise that sequence-level aggregation of information may provide more accurate content prediction when compared with the content extracted from the local window-based residue-level disorder predictors.
discon extends the capabilities of the binary predictors from  <cit>  as it provides a real-value, instead of binary, estimates of the amount of the disorder.
our solution has two key characteristics.
firstly, we use a comprehensive selection of the input information sources including sequence, evolutionary profiles generated with psi-blast, and predicted secondary structure, solvent accessibility, b-factors, signal peptides and globular domains.
the main reason to use such a diverse set of inputs is to capture different aspects/flavours of disorder  <cit> .
the selection of the first five sources is motivated by their successful use for the residue-level disorder predictions, see table  <dig>  similarly as for the ordered proteins, for which the correct folding into biologically active conformations is determined by their amino acid chain, the absence of rigid structure in the intrinsically disordered proteins or regions is also encoded in their amino acid sequences  <cit> .
the disordered regions are usually depleted in so-called order-promoting residues  while they include larger numbers of the disorder-promoting residues   <cit> .
moreover, the disorder is often observed in parts of the sequence that are characterized by low complexity, higher number of pro and charged residues, and lower amount of hydrophobic and bulky amino acids  <cit> , which motivates the use of the input protein chain.
the predicted secondary structure is useful since many of the disordered regions are characterized by lack of the secondary structure  <cit> .
these unstructured regions usually have a large solvent-accessible area  <cit>  which motivates the application of the predicted solvent accessibility.
high values of b-factors are often associated with disordered regions  <cit> , which is why we use their predictions to implement inputs to discon.
the signal peptides were previously found relevant in the context of analysis of differences between disorder predictors  <cit>  and residues in domains are less likely to be disordered and this information was used in the norsnet predictor  <cit> .
secondly, we perform a careful design of the input features that are computed based on the aggregation of values for each information source and also by combining information sources.
we also designed several customized features that quantify the size and relative location of the predicted secondary structure segments.
methods are sorted in the ascending order by their year of publication.
we empirically demonstrate that the discon's predictions are more accurate than the content extracted from the residue-level annotations generated by modern disorder predictors, including methods listed in table  <dig>  one of the potential applications of the predicted disorder content is to adjust the cut-offs used by the disorder predictors to annotate the disordered residues.
we show that these annotations can be improved when the threshold values is adjusted for each chain such that the amount of the predicted disordered amino acids matches not only the native but also the predicted content.
methods
definition of disorder
in the past casp experiments the disordered residues were defined as the amino acids that lack coordinates in their crystal structures and, in the case of the structures solved by nmr, as the amino acids that exhibit high variability within an ensemble or that were annotated by experimentalists as disordered in the remark  <dig>  <cit> .
another commonly used source for the disorder annotations is based on the experimentally-validated and biologically relevant disordered segments from the disprot database  <cit> .
we note that the assignment of the disordered regions using different experimental methods was previously shown to be potentially inconsistent  <cit> .
consequently, the disorder predictors that were developed using annotations provided with one method could lead to larger errors when used to predict annotations generated with the help of other methods  <cit> .
therefore, we created a dataset that combines the casp-defined annotations with the disprot annotations.
datasets
the proposed method was designed and tested using a dataset that was developed to validate a recent meta-predictor of disordered residues, the mfdp  <cit> .
the protein chains were collected from the protein data bank   <cit>  and the disprot  <cit>  databases.
the culled pdb list  <cit>  was used to derive a high-quality and low sequence identity subset of the pdb protein.
more specifically, only the proteins for which the structure is characterized by r-factor <  <dig>  and resolution <  <dig> Å, and that are characterized by sequence identity < 25% were kept.
we randomly selected 20% of the fully structured proteins among the resulting chains.
this is motivated by the fact that many of chains selected using the culled pdb list are annotated as ordered while a recent study shows that completely ordered proteins are not highly abundant in pdb  <cit> .
the pdb chains were combined with all  <dig> proteins from the release  <dig>  of the disprot.
the resulting dataset was filtered to reduce the pairwise sequence identity to below 25% by removing similar sequence with fewer disordered residues.
among the remaining  <dig> chains we removed four for which md failed to produce predictions; this also resulted in lack of predictions from ucon, profbval and norsnet that are bundled with the md predictions.
moreover, we improved the annotations of the disprot chains using the procedure described in  <cit> .
we applied the approach based on the sl dataset  <cit>  that combines the disorder annotations from the disprot with the annotations of disorder and order based on the corresponding structural domains that can be found in pdb.
we note that in contrast to the sl dataset that is based on the release  <dig>  of disprot, our annotations are based on the newer release  <dig> .
finally, we also removed the his-tags that are introduced to ease the crystallization.
the resulting dataset includes  <dig> chains from disprot and  <dig> from pdb.
this dataset was divided at random into two subsets, the training dataset with  <dig> chains and the test dataset with the remaining  <dig> chains.
we note that although there is some overlap between the training and test sequences , they are mostly independent at the 25% similarity level.
the training dataset was used to develop the predictor including selection of the input features and the parameterization of the prediction model, which were performed based on the 5-fold cross validation protocol.
next, our predictor that was computed using the training dataset was compared with the existing per-residue prediction methods using the test dataset.
the training and test datasets are available at http://biomine.ece.ualberta.ca/discon/.
evaluation criteria
the disorder content predictions are evaluated using three measures:
where yi ∈ y is the native and xi∈ x is predicted disorder content for the ith protein chain, avgx and avgy are the sample means of x and y, sx and sy are the sample standard deviations of x and y.
following  <cit>  the binary, per-residue disorder predictions and the per-chain predictions of disorder content  were assessed using two measures:
where tp is the number of true positives , fp denotes false positives , tn denotes true negatives , and fn denotes false negatives .
accuracy quantifies the overall success rate, i.e., fraction of correct predictions among all prediction, but since it may lead to misleading results when the dataset is unbalanced  we also use mcc.
the mcc values range between - <dig> and  <dig> and they are equal zero when all residues are predicted to be structured or to be disordered.
higher values of pcc, accuracy and mcc and lower values of mse and mae correspond to better predictions.
we also evaluated the real-value, per-residue disorder predictions based on the area under the roc curve  measure.
overview of the proposed predictor
the prediction of the disorder content is performed in three steps, see figure  <dig>  first, the input protein chain is processed through psi-blast  <cit>  to generate position specific scoring matrix  and weighted observed percentage  profiles using the ncbi's nr database downloaded on nov 19th  <dig>  which was filtered using pfilt  <cit>  to remove low-complexity regions, transmembrane regions, and coiled-coil segments.
we use psipred  <cit>  to obtain the 3-state predicted secondary structure , real-spine <dig>  <cit>  for the prediction of the relative solvent accessibility , profbval  <cit>  for the b-factor and residue flexibility predictions, and iupred  <cit>  to predict globular domains.
the former three methods use the pssm profiles as their inputs.
second, the above predictions, profiles and sequence are used to generate a set of numerical descriptors/features.
these features quantify the information encoded in each of the predictions/profiles and also between multiple predictions/profiles.
we performed feature selection to select a small subset of  <dig> features that are relevant to the prediction of the disorder content.
third, the selected descriptors are fed into a ridge regression model to generate the predictions.
feature-based encoding of the input protein sequence
the input sequence is processed to generate predictions of the 3-state ss, rsa, normalized  <cit>  real-value b-factors, binary annotation of the residue flexibility as provided by profbval in two modes, the strict and the non-strict  <cit> , binary annotation of residues that form globular domains, and sequence profiles encoded using pssm and wop.
we normalized the asa values predicted by real-spine <dig> using the maximal asa values provided in  <cit>  and we preprocessed the 3-state ss by converting the predicted helices that had <  <dig> residues into coils.
we also binarized the real-values rsa to annotate the residues as either solvent exposed when rsa >  <dig>  or buried when rsa ≤  <dig> ; this cut-off value was used in past studies  <cit> .
we also attempted to use signal peptide prediction provided by rpsp  <cit> , but these features were removed during the feature selection.
detailed description of features is provided in table a <dig> in the additional file  <dig>  we generated total of  <dig> features that are based on
- composition of amino acids
- length and relative location of predicted helix, strand and coil segments
- composition of solvent exposed residues
- composition of flexible residues and sequence segments composed of flexible residues
- number and size of sequence segments that correspond to predicted globular domains
- composition of residue predicted as signal peptides
- fusion of the information coming from multiple predictions, including ss states, solvent exposure, flexibility, and domain annotations.
we consider all combinations of two, three and four of the above predictions.
- aggregations of the sequence profiles using entropy and relative  entropy by both rows and columns of the pssm and wop
- entropy-based aggregations of the sequence profiles encoded with pssm and wop which is performed for specific amino acid types, and for residues characterized by specific ss state, solvent exposure, flexibility, and domain annotations.
we emphasize that most of the features, in particular the features that are based on the secondary structure segments, flexible sequence segments, and that combine multiple predicted properties, are novel and unique to this work.
design of the predictive model
the features were generated to comprehensively cover information that can be extracted from each predicted property, sequence and sequence profile, and their combinations.
consequently, some of these inputs may not be relevant to the prediction of the disorder content and some could be redundant with each other.
we performed two-step feature selection to find a small set of non-redundant and relevant features; the second step also includes computation and parameterization of the predictive model.
first, we remove the irrelevant and redundant features using a coarse-grained evaluation based on correlation, and next we perform a wrapper-based selection using the remaining features.
in the first step, for each feature we compute its average pcc with the disorder content  and we remove the features with average absolute pcc value <  <dig> .
we selected the  <dig>  cut-off as this value corresponds to a visible dip in the distribution of the correlation values, see figure a <dig> in the additional file  <dig>  next, we filtered the remaining  <dig> features to remove redundancy by assuring that the maximal average absolute pcc value between any pair of these features is <  <dig> .
starting with the feature that has the highest average absolute pcc with the native content, we added another feature into the set of filtered features if the average  absolute pcc between this feature and each feature which is already in the set of filtered features was <  <dig> .
in the second step, we use the remaining  <dig> features to perform wrapper-based selection in which a subset of features that results in favourable performance in prediction of the disorder content is retained.
we consider two types of predictors, ridge regression and support vector regression   <cit> .
the selection of the regression model is motivated by its successful application in several related areas, including evaluation of peptide identification  <cit>  and prediction of folding rates  <cit> , solvent accessibility  <cit> , secondary structure content  <cit> , and affinity of protein-ligand complexes  <cit> , to name a few.
similarly, the svr also enjoys a wide range of relevant applications including prediction of b-factors  <cit> , solvent accessibility  <cit> , and half-sphere exposure  <cit> .
the values of the regression coefficients and the svr models were estimated from the data in the training folds using weka workbench  <cit> .
we consider three types of kernel functions to build svr models, polynomial, radial basis function , and pearson vii function-based universal kernel   <cit> .
we parameterized each kernel and the complexity constant c by performing grid search.
we use linear and quadratic polynomials, and c equal 2x where x = - <dig>  - <dig> ..., 2; the rbf kernel with gamma  equal 2y where y = - <dig>  - <dig> ...,  <dig>  and c values where x = - <dig>  - <dig> ..., 6; and the puk kernel with omega equal 2z where z = - <dig>  - <dig> ...,  <dig>  and c values where x = - <dig>  - <dig> ...,  <dig>  we also parameterized the ridge parameter in the ridge regression; we considered ridge values equal 10w where w = - <dig>  - <dig> ...,  <dig>  we first parameterized these  <dig> predictors  using a representative subset of the  <dig> features.
we selected one features with the highest average absolute pcc from each of the feature groups defined in table a <dig> in the additional file  <dig>  the representative subset includes  <dig> features since that number of groups was covered among the  <dig> features.
next, these parameterized predictors were used to perform feature selection in which we searched for a subset of features that results in the best mse value.
we performed forward and backward best first searches.
the forward/backward best first search starts with the empty/entire  set of features, and it adds/removes one feature at the time if it decreases/increases the mse value.
the search stops when the entire list of features is scanned.
as a result, we obtained  <dig> configurations of  <dig> predictors with  <dig> search types.
the predictors in each configuration were parameterized using the grid search as described above.
the parameterizations and all steps of the feature selection were executed based on multiple repetitions of 5-fold cross validations on the training dataset, and they aimed to minimize the average mse score between the predicted and the native disorder content.
we repeated the cross validations for up to five times using randomized division into the  <dig> folds for as long as the coefficient of variation  was below  <dig> ; this approach should assure a robust estimate of the mse values.
the parameters of the four predictors and the corresponding number of the selected features are given in table a <dig> in the additional file  <dig>  the predictive performance, which was evaluated based on 5-fold cross validation on the training dataset, for the eight configurations is summarized in table a <dig> in the additional file  <dig>  the best performance, in terms of the mse and pcc values, is achieved with the ridge regression that uses  <dig> features selected using the forward best first search, and this configuration is used to implement the proposed discon predictor.
RESULTS
disorder content prediction
we compare the performance of the discon with the results obtained using the disorder content computed from the disorder predictions generated by disopred <dig>  iupred , profbval, norsnet, ucon, disoclust, md, pondr-fit, and mfdp methods.
for the per-residue predictors we used the web servers or standalone implementations provided by the authors, and we calculated the content by counting the number of residues predicted as disordered and dividing it by the length of the corresponding protein chain.
the results are computed on the test dataset with  <dig> chains which shares low identity to chains in our training dataset.
we note that the methods we compare with use training datasets that may share higher similarity with the chains in our dataset, which could inflate their predictive quality.
we also analyze statistical significance of the differences between the content predicted by discon and the other methods.
we compare the per-chain values of the absolute errors and the squared errors over the  <dig> chains in the test dataset and the pearson correlation coefficients computed for  <dig> randomly selected sets of  <dig> proteins from the test dataset.
since the measurements follow normal distribution  we apply the paired t-test and we measure significance of the differences at  <dig>  and  <dig>  levels.
we evaluate the extent of the over-and under-prediction of the disorder content by quantifying the number of the over-and under-predicted chains and the corresponding mae values and we also provide the auc, accuracy, and mcc values for the  <dig> considered per-residue predictors.
the results are summarized in table  <dig>
the discon is shown to provide favourable predictive performance.
it obtains mse equal  <dig>  and pcc equal  <dig>  on the test dataset.
we note that these results are consistent with the results obtained based on the 5-fold cross validation on the training dataset .
on the test set, the best performing per-residue disorder predictors are worse than discon by  <dig>  mse and  <dig>  pcc for the disorder content prediction.
the average absolute error of discon equals  <dig>  when compared with value at or over  <dig>  obtained with the current disorder predictors, except for iupreds for which mae =  <dig> .
the improvements in mse and pcc offered by discon are shown to be statistically significant when compared with all considered competitors.
the mae values computed from our predictions are significantly better than the errors based on the predictions with four existing methods and are equivalent with the remaining six predictors.
further analysis reveals that the quality of the discon predictions is better for longer chains, while some other methods may produce favorable predictions for short chains.
figure a <dig> in the additional file  <dig>  which shows the relation between the chain length and the absolute errors generated by the discon and the top-three methods from table  <dig>  i.e., iupreds, ucon, and pondr-fit, demonstrates that the proposed predictor is characterized by smaller absolute errors for longer chains, while the other three predictors on average provide more accurate predictions for short chains.
discon provides relatively balanced predictions with similar number of over-and under-predicted chains and low mae values for these two types of errors.
we observe that profbval and disoclust are characterized by substantial levels of the over-prediction of the disorder content that are expressed by the large number of the over-predicted chains and/or high mae for the over-predicted chains.
the under-prediction of the disorder content is characteristic for the norsnet method.
table  <dig> also shows that the ucon which obtains relatively low mse and mid-range pcc is characterized by lower quality of the per-residue predictions with mcc =  <dig> .
binary prediction of the disorder amount
we apply the predicted disorder content on the test dataset to perform binary prediction of chains that are characterized by the amount of the disorder below/above a specific cut-off value.
the cut-offs at  <dig> and  <dig> corresponds to detection of fully-disordered and fully-ordered proteins, respectively, while the intermediate cut-off values could be used to find partially structured chains.
we measure the mcc of these binary predictions defined using the cut-off values between  <dig> and  <dig> with step of  <dig> , and the content predicted by discon and extracted from the outputs of the  <dig> considered disorder predictors, see figure  <dig>  we observe that the usage of the content predicted by discon results in the highest mcc values, which range  <dig>  and  <dig> , for cut-offs between  <dig>  and  <dig>  inclusive.
the best results for the cut-off values above  <dig>  are achieved with the content predicted with md while several predictors do comparably well for the cut-offs below  <dig> .
we note that the prediction of the fully-structured and fully-disordered proteins could be also accomplished using specialized predictors, such as nn-cdf  <cit> .
content guided thresholding of the real-value disorder prediction
the disorder predictors usually provide both the real-value propensity of the disorder and the binary order/disorder assignment for each residue.
the binary assignment is usually based on thresholding of the real-value propensities with a fixed cut-off.
we investigate whether the predicted disorder content could be used to guide the selection of the threshold value.
this means that instead of using the fixed cut-off we adjust the threshold such that the amount of the residues annotated as disordered equals to the amount of the disorder content predicted with discon .
we evaluate the binary residue-level disorder prediction of the original and the content-adjusted disorder predictors using mcc, see figure  <dig>  and accuracy, see figure a <dig> in the additional file  <dig>  we note that the  <dig> methods that are considered in these figures provide continuous  prediction values that represent the propensity for a residue to be disordered.
we compare the original binary disorder predictions, predictions that are based on a fixed cut-off that maximizes the mcc of a given predictor on our test dataset , and predictions where the cut-off is selected to match the content predicted by discon.
based on the figure  <dig> that shows that md outperforms discon when the native content <  <dig>  or >  <dig> , we also consider combining content predicted by the md and discon to adjust the threshold.
if the content predicted by md is >  <dig>  or <  <dig>  for a given chain then we use the md predicted content; otherwise we use the content predicted by the discon method.
the thresholding of the predicted real-value disorder using the content predicted by discon leads to improvements in both mcc and accuracy for all predictors except for the md, in which case the accuracy is slightly improved but the mcc is lower.
the average  improvement in mcc and accuracy equal  <dig>  and  <dig> , respectively.
when we use the combination of the content predicted with discon and md the improvements are more substantial and they range between  <dig>  and  <dig>  for the mcc  and  <dig>  and  <dig>  for the accuracy ; the best mcc is obtained using the predictions from the mfdp and it equals  <dig>  when compared with  <dig>  that was obtained without the content-based adjustment.
interestingly, using this cut-off adjustment the mcc values obtained by seven out of ten considered predictors are >  <dig>  while originally , see table  <dig>  only two methods have mcc >  <dig> .
this suggests that majority of the considered disorder predictors differentiate between structured and disordered residues based on their real-value propensities in a given chain with relatively similar quality, but only a few of them can accurately scale the range of the real-value propensities between sequences.
the content-guided selection of the cut-offs alleviates the prediction bias, i.e., the tendency to under-or over-predict the amount of disorder.
the binary predictions of profbval, disoclust, and norsnet that are originally characterized by relatively low mcc values and a bias towards either over-or under-prediction, see table  <dig>  are shown to improve by a wide margin when using the disorder content predicted by discon or by the combination of discon and md.
we observe that the relatively poor performance of the ucon method does not stem from the prediction bias but rather from its overall problems with the quality of the residue-level annotations, as evidenced by the relatively low auc and mcc in table  <dig> which is in contrast to the sequence-level amount of disorder that is predicted quite accurately by this method.
we visualize the improvements that result from the cut-off adjustments using two case studies, one where the original predictions over-estimate the native amount of disorder and another where the predictions are under-estimated.
in both cases, we compare the original binary annotations of disordered residues with the annotations that are adjusted using the content predicted with disccon, and we include predictions from the top six methods from table  <dig>  i.e., ucon, md, mfdp, pondr-fit, iupreds, and disopred <dig>  the first example is the apoptosis-inducing ligand  <dig>  protein , see figure  <dig>  this protein was solved using high-resolution  x-ray crystallography and the structure includes two relatively short disordered segments in the vicinity of the n-terminus  <cit> .
figure  <dig> reveals that all six predictors annotate disorder at the n-terminus and that pondr-fit and iupreds also predict a short disordered segment at the c-terminus.
however, the disorder at the n-terminus is over-predicted; the residues between positions  <dig> and  <dig> and between  <dig> to about  <dig> are predicted as disordered, while the x-ray structure shows them as structured.
these over-predictions were minimized when the cut-off was adjusted to match the content predicted by the discon.
importantly, the adjusted predictions identify the two disordered segments, with particularly good results for the md and mfdp predictors that quite accurately identify both of the disordered segments and no other disordered residues.
after the adjustment, the predictions from pondr-fit show the two disordered segments at the n-terminus, although the first segment is predicted to be  <dig> residues too short, and the c-terminus is predicted as structured.
similarly, the adjusted predictions from iupreds and disopred <dig> show disorder in the vicinity of the n-terminus, while the disorder predictions in other parts of this chain are removed.
the second case is the inosine-5'-monophosphate dehydrogenase  which was also solved using x-ray crystallography, see figure  <dig>  this protein includes four disordered segments, one longer between positions  <dig> and  <dig>  and three shorter towards the c-terminus  <cit> .
overall, the six predictors under-predicted the disorder levels in this protein.
they predicted only a few disordered residues at both termini, with the exception of ucon that predicted about a dozen of short disordered segments throughout the entire chain and mfdp that predicted three disordered segments, including both termini and a segment between positions  <dig> and  <dig>  the discon predicted  <dig> % of residues to be disordered, when compared with 1%,  <dig> %,  <dig> %, 3%,  <dig> %, and 12% that were predicted by the disopred <dig>  md, iupreds, pondr-fit, mfdp, and ucon, respectively; the native amount of disorder is  <dig> %.
the content-adjusted annotation of the disordered residues captures a large number of disordered residues in the long segment between positions  <dig> and  <dig>  as well as the two disordered segments nearest to the c-terminus.
these improvements come as a trade-off for an over-prediction of the disorder at the n-terminus, particularly for the pondr-fit and disopred <dig> predictors.
overall, we observe that the adjusted predictions show a denser concentration of the disordered residues around the natively disordered regions.
we conclude that although predictions shown in the two case studies should not be assumed typical, they demonstrate that the content predicted with discon offers valuable assistance in selection of the cut-offs to annotate the disordered residues based on the real-values predictions from modern disorder predictors.
factors related to the amount of disorder/order
we convert the input protein sequence into a custom-designed set of selected  <dig> numerical descriptors which utilize information related to the evolutionary profiles, sequence itself, and predicted secondary structure , solvent accessible residues , b-factors and globular domains; see table a <dig> in the additional file  <dig>  majority of the selected features combine multiple input sources.
for instance, the largest group of  <dig> similar features is based on counting the residues in certain predicted ss states, with certain levels of predicted solvent exposure and b-factors which are located within the predicted domains.
for instance, the sshedominbfnslowrsab feature counts the predicted helix and strand residues that are in globular domains, have low b-factors  and are buried.
as expected, these values are negatively correlated with the content of the native disorder , see figure  <dig>  since this descriptor highlights hallmarks of well-structured proteins, i.e., they usually include conserved domains with buried strands and/or helices that are usually structurally rigid.
another feature that attains negative, - <dig> , correlation with the native content is bfnslowseg <dig>  which quantifies the number of predicted rigid residues  that are grouped together in the sequence in segments of size at least  <dig>  figure  <dig> shows a cluster of bfnslowseg <dig> values between  <dig>  and  <dig>  for chains with low amount of native disorder, which suggests that well-structured proteins include significant amount of rigid residues that are grouped together in the sequence, while disordered chains contain fewer of such rigid residues.
we also discuss two features that have relatively high positive correlation with the disorder content.
the sschbfnshighdomnotin feature counts the number of coil and helix residues with high b-factor that are not in the globular domains.
this feature that has pcc =  <dig>  in our dataset, see figure  <dig>  agrees with characteristic properties of the disorder, which often concerns flexible residues that are outside of the domains.
the chc...chseg feature  computes the number of residues in the longest segment in the predicted ss that does not include strands, i.e., stretch of the sequence that is composed only of a coils or of alternating helix-coil-helix-coil... segments.
this feature demonstrates that disordered proteins are often depleted of beta-sheets.
the input features also highlight the importance of the relation between sequence conservation and the amount of the disorder, i.e.,  <dig> out of the  <dig> features utilize entropy computed from the pssm or wop profiles.
for instance, the entavepssm feature, which computes the entropy of the average pssm scores for each column  in the matrix along the sequence, has pcc = - <dig> .
this means that well-structured proteins are characterized on average by a stronger degree of sequence conservation when compared with the disordered proteins.
our observation is in agreement with the results of previous study, where the evolution rates of ordered and intrinsically disordered regions were compared using the pairwise genetic distances between the ordered and the disordered regions of  <dig> protein families having at least one member with a structurally characterized region of disorder of  <dig> or more consecutive residues  <cit> .
this study demonstrated that the disordered regions evolved significantly more rapidly than the ordered regions in  <dig> of the  <dig> families studied  <cit> .
CONCLUSIONS
in spite of the fact that the quality of the high-throughput disorder prediction continues to improve  <cit> , researchers recognize that new and more accurate predictors are still needed  <cit> .
we address the shortage of accurate methods that predict the overall amount of disorder in a given protein chain, which is motivated by the fact that current disorder predictors tend to provide relatively inaccurate estimates of the disorder content.
we propose a novel approach, called discon, which combines information derived from sequence, sequence profiles, and predicted secondary structure, solvent accessibility, flexibility, and annotation of globular domains.
we custom designed feature-based representation of the input protein chain that aggregates and combines these inputs and we performed feature selection that found a small set of  <dig> complementary features that are well correlated with the native disorder.
using these features and a ridge regression-based model, the discon predicts the disorder content with low,  <dig> , mean squared error and high,  <dig> , correlation, as evaluated on an independent test dataset.
these predictions are empirically shown to be significantly better than the disorder content estimates derived from outputs of ten modern disorder predictors.
the discon's predictions provide a high-quality alternative for high-throughput annotation of the disorder content.
they are also shown to provide useful input to improve binary annotations of the disordered residues from the real-value disorder propensities generated by current disorder prediction methods.
list of abbreviations
auc: area under the roc curve; asa: absolute solvent accessibility; casp: critical assessment of techniques for protein structure prediction; cdf: cumulative distribution function; ch-plot: charge-hydropathy plot; discon: disorder content predictor; idp: intrinsically disordered protein; mae: mean absolute error; mcc: matthews correlation coefficient; mse: mean squared error; nmr: nuclear magnetic resonance; pcc: pearson correlation coefficient; pdb: protein data bank; pssm: position specific scoring matrix; puk: pearson vii function-based universal kernel; rbf: radial basis function; rsa: relative solvent accessibility; ss: secondary structure; svr: support vector regression; wop: weighted observed percentage.
competing interests
the authors declare that they have no competing interests.
authors' contributions
mjm contributed to the collection of the data, design of the prediction method, experimental validation, analysis and interpretation of the results, and implementation of the web server.
tz worked on the design and implementation of the prediction method and analysis and interpretation of the results.
bx helped with the analysis and interpretation of the results.
yz and akd helped with the conception of the project and analysis and interpretation of the results.
vnu contributed to the analysis and interpretation of the results and writing of the manuscript.
lk was responsible for the conception of the project, and contributed to the collection of the data, design and implementation of the prediction method, experimental validation, analysis and interpretation of the results, and writing of the manuscript.
all authors have read, corrected and approved the manuscript.
supplementary material
additional file 1
supplementary tables and figures.
this file includes  <dig> supplementary tables and  <dig> supplementary figures.
the tables summarize the input features and results obtained with alternative designs of the proposed predictor.
the figures summarize correlation between the input features and the predictive target, the relation between the predictive quality and the input chain length, and the accuracy for the residue-level disorder predictions.
click here for file
acknowledgements
this work was sponsored in part by the discovery grant from nserc canada to lk, the national institutes of health grant  to yz, and the killam memorial scholarship to mjm.
the funding agencies did not participate in the design, collection, analysis, and interpretation of the data.

 the intrinsically disordered proteins, also referred to as natively unfolded or intrinsically unstructured proteins, lack stable tertiary structure in vitro. these proteins are implicated in numerous processes including cellular signal transduction, transcriptional regulation, and translation, and their prevalence was demonstrated in several human diseases, including cancer, cardiovascular disease, neurodegenerative diseases, genetic diseases, and amyloidoses. at the same time, the annotations of the idps are accumulated at a relatively low pace when compared with the growth of the number of known, non-redundant protein sequences. over the last decade numerous sequence-derived characteristics, including low complexity, which was proposed in, high net charge and low content of hydrophobic amino acids, lack of regular secondary structure, to name just a few, were found to differentiate between disordered and ordered regions. the abovementioned results suggest that disorder can be predicted from the sequence and they motivate the development of computational models for the prediction of the disordered regions. several such predictors were already developed; see for a recent review. majority of the existing predictors generate the disorder predictions for each residue in the input protein chain. these per-residue predictors can be divided into categories: i) methods that utilize the relative propensity of amino acids to form disorder/ordered regions which include globplot, iupred, foldindex, and ucon; ii) methods that are based on classifiers generated with the help of machine learning algorithms, such as dispro, disopred, disopred, prdos, poodle predictors, pondr predictors, spritz, profbval, dispssmp, dispssmp, iup, norsnet and ond-crfs; iii) meta-approach methods that are based on a consensus of multiple base predictors including multicom, metaprdos and recent md, mfdp, and pondr-fit predictors; and iv) approaches that find disordered residues through an analysis of the predicted 3d structural models such as prdos and disoclust. there are also methods that predict the propensity of the entire protein chain to be unstructured. one of these approaches is based on the charge-hydropathy plots and another utilizes distributions of the predicted per-residue disorder scores. the abovementioned per-residue and per-chain methods perform the predictions in a high-throughput manner and consequently they can be used as a possible solution to close the annotation gap. although the per-residue methods are successful in the disorder prediction at the residue level, i.e., they achieve auc of about and mcc of about when tested on large benchmark datasets, we observe that they typically make relatively substantial mistakes at the sequence-level. more specifically, these methods may over-or under-predict the overall amount of disorder in the sequence. tests of recent disorder predictors that include methods from all four groups on a benchmark dataset of chains, see table show that the mean average squared errors between the native and the predicted amount of disorder vary between and. one of the potential reasons for these errors is the fact that virtually all of the most accurate recent predictors, such as norsnet, disopred md, pondr-fit, and mfdp, use a local sequence window to predict the disorder while the information encoded in the entire chain may reveal an overall sequence-level disorder bias. the disorder predictors use identical cut-off values to annotate disordered residues based on the predicted real-value propensities, and we show that these annotations can be improved if the cut-off is adjusted to match the native amount of the disorder in the entire chain, which suggests that the knowledge of this sequence bias could be useful. we report the mse, mae, and pcc values, the percentage of chains that are over-predicted and under-predicted, and the mae value for the over-and under-predicted chains. the methods are sorted in the descending order by the mse values and the best values are shown in bold font. results of the tests of significance of the differences between discon and the other methods are given in the "stat. signif." columns. the tests compare the absolute and the squared errors per-chain over all chains in our test dataset, and pearson correlation computed for randomly selected sets of proteins from the test dataset. the ++ and + denote that discon is statistically significantly better with p < and p <, respectively, and = denotes that the results are not significantly different. we also report their area under curve, accuracy and mcc for the per-residue disorder predictions generated by the ten considered predictors. the overall disorder content was used in the past to estimate the abundance of intrinsic disorder in several protein databases, in various protein families and classes, and in complete proteomes. the high values of the disorder content were reported for several disease-related proteins. the content was also used for the analysis of intrinsic disorder-related protein functions. importantly, in all these and similar cases, the disorder content was evaluated based on the results of either binary classifiers or was derived from the per-residue disorder predictions. as mentioned above, these per-residue disorder prediction methods may over-or under-predict the overall amount of disorder in the sequence. this observation and the fact that the knowledge of the disorder content in a given protein or in a set of proteins of interest or in an entire proteome can be utilized to investigate numerous important hypotheses motivate the development of new computational tools for the accurate prediction of the disorder content. we propose a novel method, named discon, that aims to provide accurate sequence-based predictions of the disorder content. our approach is based on the premise that sequence-level aggregation of information may provide more accurate content prediction when compared with the content extracted from the local window-based residue-level disorder predictors. discon extends the capabilities of the binary predictors from as it provides a real-value, instead of binary, estimates of the amount of the disorder. our solution has two key characteristics. firstly, we use a comprehensive selection of the input information sources including sequence, evolutionary profiles generated with psi-blast, and predicted secondary structure, solvent accessibility, b-factors, signal peptides and globular domains. the main reason to use such a diverse set of inputs is to capture different aspects/flavours of disorder. the selection of the first five sources is motivated by their successful use for the residue-level disorder predictions, see table similarly as for the ordered proteins, for which the correct folding into biologically active conformations is determined by their amino acid chain, the absence of rigid structure in the intrinsically disordered proteins or regions is also encoded in their amino acid sequences. the disordered regions are usually depleted in so-called order-promoting residues while they include larger numbers of the disorder-promoting residues. moreover, the disorder is often observed in parts of the sequence that are characterized by low complexity, higher number of pro and charged residues, and lower amount of hydrophobic and bulky amino acids, which motivates the use of the input protein chain. the predicted secondary structure is useful since many of the disordered regions are characterized by lack of the secondary structure. these unstructured regions usually have a large solvent-accessible area which motivates the application of the predicted solvent accessibility. high values of b-factors are often associated with disordered regions, which is why we use their predictions to implement inputs to discon. the signal peptides were previously found relevant in the context of analysis of differences between disorder predictors and residues in domains are less likely to be disordered and this information was used in the norsnet predictor. secondly, we perform a careful design of the input features that are computed based on the aggregation of values for each information source and also by combining information sources. we also designed several customized features that quantify the size and relative location of the predicted secondary structure segments. methods are sorted in the ascending order by their year of publication. we empirically demonstrate that the discon's predictions are more accurate than the content extracted from the residue-level annotations generated by modern disorder predictors, including methods listed in table one of the potential applications of the predicted disorder content is to adjust the cut-offs used by the disorder predictors to annotate the disordered residues. we show that these annotations can be improved when the threshold values is adjusted for each chain such that the amount of the predicted disordered amino acids matches not only the native but also the predicted content. methods definition of disorder in the past casp experiments the disordered residues were defined as the amino acids that lack coordinates in their crystal structures and, in the case of the structures solved by nmr, as the amino acids that exhibit high variability within an ensemble or that were annotated by experimentalists as disordered in the remark. another commonly used source for the disorder annotations is based on the experimentally-validated and biologically relevant disordered segments from the disprot database. we note that the assignment of the disordered regions using different experimental methods was previously shown to be potentially inconsistent. consequently, the disorder predictors that were developed using annotations provided with one method could lead to larger errors when used to predict annotations generated with the help of other methods. therefore, we created a dataset that combines the casp-defined annotations with the disprot annotations. datasets the proposed method was designed and tested using a dataset that was developed to validate a recent meta-predictor of disordered residues, the mfdp. the protein chains were collected from the protein data bank and the disprot databases. the culled pdb list was used to derive a high-quality and low sequence identity subset of the pdb protein. more specifically, only the proteins for which the structure is characterized by r-factor < and resolution < Å, and that are characterized by sequence identity < 25% were kept. we randomly selected 20% of the fully structured proteins among the resulting chains. this is motivated by the fact that many of chains selected using the culled pdb list are annotated as ordered while a recent study shows that completely ordered proteins are not highly abundant in pdb. the pdb chains were combined with all proteins from the release of the disprot. the resulting dataset was filtered to reduce the pairwise sequence identity to below 25% by removing similar sequence with fewer disordered residues. among the remaining chains we removed four for which md failed to produce predictions; this also resulted in lack of predictions from ucon, profbval and norsnet that are bundled with the md predictions. moreover, we improved the annotations of the disprot chains using the procedure described in. we applied the approach based on the sl dataset that combines the disorder annotations from the disprot with the annotations of disorder and order based on the corresponding structural domains that can be found in pdb. we note that in contrast to the sl dataset that is based on the release of disprot, our annotations are based on the newer release. finally, we also removed the his-tags that are introduced to ease the crystallization. the resulting dataset includes chains from disprot and from pdb. this dataset was divided at random into two subsets, the training dataset with chains and the test dataset with the remaining chains. we note that although there is some overlap between the training and test sequences, they are mostly independent at the 25% similarity level. the training dataset was used to develop the predictor including selection of the input features and the parameterization of the prediction model, which were performed based on the 5-fold cross validation protocol. next, our predictor that was computed using the training dataset was compared with the existing per-residue prediction methods using the test dataset. the training and test datasets are available at http://biomine.ece.ualberta.ca/discon/. evaluation criteria the disorder content predictions are evaluated using three measures: where yi ∈ y is the native and xi∈ x is predicted disorder content for the ith protein chain, avgx and avgy are the sample means of x and y, sx and sy are the sample standard deviations of x and y. following the binary, per-residue disorder predictions and the per-chain predictions of disorder content were assessed using two measures: where tp is the number of true positives, fp denotes false positives, tn denotes true negatives, and fn denotes false negatives. accuracy quantifies the overall success rate, i.e., fraction of correct predictions among all prediction, but since it may lead to misleading results when the dataset is unbalanced we also use mcc. the mcc values range between - and and they are equal zero when all residues are predicted to be structured or to be disordered. higher values of pcc, accuracy and mcc and lower values of mse and mae correspond to better predictions. we also evaluated the real-value, per-residue disorder predictions based on the area under the roc curve measure. overview of the proposed predictor the prediction of the disorder content is performed in three steps, see figure first, the input protein chain is processed through psi-blast to generate position specific scoring matrix and weighted observed percentage profiles using the ncbi's nr database downloaded on nov 19th which was filtered using pfilt to remove low-complexity regions, transmembrane regions, and coiled-coil segments. we use psipred to obtain the 3-state predicted secondary structure, real-spine for the prediction of the relative solvent accessibility, profbval for the b-factor and residue flexibility predictions, and iupred to predict globular domains. the former three methods use the pssm profiles as their inputs. second, the above predictions, profiles and sequence are used to generate a set of numerical descriptors/features. these features quantify the information encoded in each of the predictions/profiles and also between multiple predictions/profiles. we performed feature selection to select a small subset of features that are relevant to the prediction of the disorder content. third, the selected descriptors are fed into a ridge regression model to generate the predictions. feature-based encoding of the input protein sequence the input sequence is processed to generate predictions of the 3-state ss, rsa, normalized real-value b-factors, binary annotation of the residue flexibility as provided by profbval in two modes, the strict and the non-strict, binary annotation of residues that form globular domains, and sequence profiles encoded using pssm and wop. we normalized the asa values predicted by real-spine using the maximal asa values provided in and we preprocessed the 3-state ss by converting the predicted helices that had < residues into coils. we also binarized the real-values rsa to annotate the residues as either solvent exposed when rsa > or buried when rsa ≤; this cut-off value was used in past studies. we also attempted to use signal peptide prediction provided by rpsp, but these features were removed during the feature selection. detailed description of features is provided in table a in the additional file we generated total of features that are based on - composition of amino acids - length and relative location of predicted helix, strand and coil segments - composition of solvent exposed residues - composition of flexible residues and sequence segments composed of flexible residues - number and size of sequence segments that correspond to predicted globular domains - composition of residue predicted as signal peptides - fusion of the information coming from multiple predictions, including ss states, solvent exposure, flexibility, and domain annotations. we consider all combinations of two, three and four of the above predictions. - aggregations of the sequence profiles using entropy and relative entropy by both rows and columns of the pssm and wop - entropy-based aggregations of the sequence profiles encoded with pssm and wop which is performed for specific amino acid types, and for residues characterized by specific ss state, solvent exposure, flexibility, and domain annotations. we emphasize that most of the features, in particular the features that are based on the secondary structure segments, flexible sequence segments, and that combine multiple predicted properties, are novel and unique to this work. design of the predictive model the features were generated to comprehensively cover information that can be extracted from each predicted property, sequence and sequence profile, and their combinations. consequently, some of these inputs may not be relevant to the prediction of the disorder content and some could be redundant with each other. we performed two-step feature selection to find a small set of non-redundant and relevant features; the second step also includes computation and parameterization of the predictive model. first, we remove the irrelevant and redundant features using a coarse-grained evaluation based on correlation, and next we perform a wrapper-based selection using the remaining features. in the first step, for each feature we compute its average pcc with the disorder content and we remove the features with average absolute pcc value <. we selected the cut-off as this value corresponds to a visible dip in the distribution of the correlation values, see figure a in the additional file next, we filtered the remaining features to remove redundancy by assuring that the maximal average absolute pcc value between any pair of these features is <. starting with the feature that has the highest average absolute pcc with the native content, we added another feature into the set of filtered features if the average absolute pcc between this feature and each feature which is already in the set of filtered features was <. in the second step, we use the remaining features to perform wrapper-based selection in which a subset of features that results in favourable performance in prediction of the disorder content is retained. we consider two types of predictors, ridge regression and support vector regression. the selection of the regression model is motivated by its successful application in several related areas, including evaluation of peptide identification and prediction of folding rates, solvent accessibility, secondary structure content, and affinity of protein-ligand complexes, to name a few. similarly, the svr also enjoys a wide range of relevant applications including prediction of b-factors, solvent accessibility, and half-sphere exposure. the values of the regression coefficients and the svr models were estimated from the data in the training folds using weka workbench. we consider three types of kernel functions to build svr models, polynomial, radial basis function, and pearson vii function-based universal kernel. we parameterized each kernel and the complexity constant c by performing grid search. we use linear and quadratic polynomials, and c equal 2x where x = - -..., 2; the rbf kernel with gamma equal 2y where y = - -..., and c values where x = - -..., 6; and the puk kernel with omega equal 2z where z = - -..., and c values where x = - -..., we also parameterized the ridge parameter in the ridge regression; we considered ridge values equal 10w where w = - -..., we first parameterized these predictors using a representative subset of the features. we selected one features with the highest average absolute pcc from each of the feature groups defined in table a in the additional file the representative subset includes features since that number of groups was covered among the features. next, these parameterized predictors were used to perform feature selection in which we searched for a subset of features that results in the best mse value. we performed forward and backward best first searches. the forward/backward best first search starts with the empty/entire set of features, and it adds/removes one feature at the time if it decreases/increases the mse value. the search stops when the entire list of features is scanned. as a result, we obtained configurations of predictors with search types. the predictors in each configuration were parameterized using the grid search as described above. the parameterizations and all steps of the feature selection were executed based on multiple repetitions of 5-fold cross validations on the training dataset, and they aimed to minimize the average mse score between the predicted and the native disorder content. we repeated the cross validations for up to five times using randomized division into the folds for as long as the coefficient of variation was below; this approach should assure a robust estimate of the mse values. the parameters of the four predictors and the corresponding number of the selected features are given in table a in the additional file the predictive performance, which was evaluated based on 5-fold cross validation on the training dataset, for the eight configurations is summarized in table a in the additional file the best performance, in terms of the mse and pcc values, is achieved with the ridge regression that uses features selected using the forward best first search, and this configuration is used to implement the proposed discon predictor.  disorder content prediction we compare the performance of the discon with the results obtained using the disorder content computed from the disorder predictions generated by disopred iupred, profbval, norsnet, ucon, disoclust, md, pondr-fit, and mfdp methods. for the per-residue predictors we used the web servers or standalone implementations provided by the authors, and we calculated the content by counting the number of residues predicted as disordered and dividing it by the length of the corresponding protein chain. the results are computed on the test dataset with chains which shares low identity to chains in our training dataset. we note that the methods we compare with use training datasets that may share higher similarity with the chains in our dataset, which could inflate their predictive quality. we also analyze statistical significance of the differences between the content predicted by discon and the other methods. we compare the per-chain values of the absolute errors and the squared errors over the chains in the test dataset and the pearson correlation coefficients computed for randomly selected sets of proteins from the test dataset. since the measurements follow normal distribution we apply the paired t-test and we measure significance of the differences at and levels. we evaluate the extent of the over-and under-prediction of the disorder content by quantifying the number of the over-and under-predicted chains and the corresponding mae values and we also provide the auc, accuracy, and mcc values for the considered per-residue predictors. the results are summarized in table the discon is shown to provide favourable predictive performance. it obtains mse equal and pcc equal on the test dataset. we note that these results are consistent with the results obtained based on the 5-fold cross validation on the training dataset. on the test set, the best performing per-residue disorder predictors are worse than discon by mse and pcc for the disorder content prediction. the average absolute error of discon equals when compared with value at or over obtained with the current disorder predictors, except for iupreds for which mae =. the improvements in mse and pcc offered by discon are shown to be statistically significant when compared with all considered competitors. the mae values computed from our predictions are significantly better than the errors based on the predictions with four existing methods and are equivalent with the remaining six predictors. further analysis reveals that the quality of the discon predictions is better for longer chains, while some other methods may produce favorable predictions for short chains. figure a in the additional file which shows the relation between the chain length and the absolute errors generated by the discon and the top-three methods from table i.e., iupreds, ucon, and pondr-fit, demonstrates that the proposed predictor is characterized by smaller absolute errors for longer chains, while the other three predictors on average provide more accurate predictions for short chains. discon provides relatively balanced predictions with similar number of over-and under-predicted chains and low mae values for these two types of errors. we observe that profbval and disoclust are characterized by substantial levels of the over-prediction of the disorder content that are expressed by the large number of the over-predicted chains and/or high mae for the over-predicted chains. the under-prediction of the disorder content is characteristic for the norsnet method. table also shows that the ucon which obtains relatively low mse and mid-range pcc is characterized by lower quality of the per-residue predictions with mcc =. binary prediction of the disorder amount we apply the predicted disorder content on the test dataset to perform binary prediction of chains that are characterized by the amount of the disorder below/above a specific cut-off value. the cut-offs at and corresponds to detection of fully-disordered and fully-ordered proteins, respectively, while the intermediate cut-off values could be used to find partially structured chains. we measure the mcc of these binary predictions defined using the cut-off values between and with step of, and the content predicted by discon and extracted from the outputs of the considered disorder predictors, see figure we observe that the usage of the content predicted by discon results in the highest mcc values, which range and, for cut-offs between and inclusive. the best results for the cut-off values above are achieved with the content predicted with md while several predictors do comparably well for the cut-offs below. we note that the prediction of the fully-structured and fully-disordered proteins could be also accomplished using specialized predictors, such as nn-cdf. content guided thresholding of the real-value disorder prediction the disorder predictors usually provide both the real-value propensity of the disorder and the binary order/disorder assignment for each residue. the binary assignment is usually based on thresholding of the real-value propensities with a fixed cut-off. we investigate whether the predicted disorder content could be used to guide the selection of the threshold value. this means that instead of using the fixed cut-off we adjust the threshold such that the amount of the residues annotated as disordered equals to the amount of the disorder content predicted with discon. we evaluate the binary residue-level disorder prediction of the original and the content-adjusted disorder predictors using mcc, see figure and accuracy, see figure a in the additional file we note that the methods that are considered in these figures provide continuous prediction values that represent the propensity for a residue to be disordered. we compare the original binary disorder predictions, predictions that are based on a fixed cut-off that maximizes the mcc of a given predictor on our test dataset, and predictions where the cut-off is selected to match the content predicted by discon. based on the figure that shows that md outperforms discon when the native content < or >, we also consider combining content predicted by the md and discon to adjust the threshold. if the content predicted by md is > or < for a given chain then we use the md predicted content; otherwise we use the content predicted by the discon method. the thresholding of the predicted real-value disorder using the content predicted by discon leads to improvements in both mcc and accuracy for all predictors except for the md, in which case the accuracy is slightly improved but the mcc is lower. the average improvement in mcc and accuracy equal and, respectively. when we use the combination of the content predicted with discon and md the improvements are more substantial and they range between and for the mcc and and for the accuracy; the best mcc is obtained using the predictions from the mfdp and it equals when compared with that was obtained without the content-based adjustment. interestingly, using this cut-off adjustment the mcc values obtained by seven out of ten considered predictors are > while originally, see table only two methods have mcc >. this suggests that majority of the considered disorder predictors differentiate between structured and disordered residues based on their real-value propensities in a given chain with relatively similar quality, but only a few of them can accurately scale the range of the real-value propensities between sequences. the content-guided selection of the cut-offs alleviates the prediction bias, i.e., the tendency to under-or over-predict the amount of disorder. the binary predictions of profbval, disoclust, and norsnet that are originally characterized by relatively low mcc values and a bias towards either over-or under-prediction, see table are shown to improve by a wide margin when using the disorder content predicted by discon or by the combination of discon and md. we observe that the relatively poor performance of the ucon method does not stem from the prediction bias but rather from its overall problems with the quality of the residue-level annotations, as evidenced by the relatively low auc and mcc in table which is in contrast to the sequence-level amount of disorder that is predicted quite accurately by this method. we visualize the improvements that result from the cut-off adjustments using two case studies, one where the original predictions over-estimate the native amount of disorder and another where the predictions are under-estimated. in both cases, we compare the original binary annotations of disordered residues with the annotations that are adjusted using the content predicted with disccon, and we include predictions from the top six methods from table i.e., ucon, md, mfdp, pondr-fit, iupreds, and disopred the first example is the apoptosis-inducing ligand protein, see figure this protein was solved using high-resolution x-ray crystallography and the structure includes two relatively short disordered segments in the vicinity of the n-terminus. figure reveals that all six predictors annotate disorder at the n-terminus and that pondr-fit and iupreds also predict a short disordered segment at the c-terminus. however, the disorder at the n-terminus is over-predicted; the residues between positions and and between to about are predicted as disordered, while the x-ray structure shows them as structured. these over-predictions were minimized when the cut-off was adjusted to match the content predicted by the discon. importantly, the adjusted predictions identify the two disordered segments, with particularly good results for the md and mfdp predictors that quite accurately identify both of the disordered segments and no other disordered residues. after the adjustment, the predictions from pondr-fit show the two disordered segments at the n-terminus, although the first segment is predicted to be residues too short, and the c-terminus is predicted as structured. similarly, the adjusted predictions from iupreds and disopred show disorder in the vicinity of the n-terminus, while the disorder predictions in other parts of this chain are removed. the second case is the inosine-5'-monophosphate dehydrogenase which was also solved using x-ray crystallography, see figure this protein includes four disordered segments, one longer between positions and and three shorter towards the c-terminus. overall, the six predictors under-predicted the disorder levels in this protein. they predicted only a few disordered residues at both termini, with the exception of ucon that predicted about a dozen of short disordered segments throughout the entire chain and mfdp that predicted three disordered segments, including both termini and a segment between positions and the discon predicted % of residues to be disordered, when compared with 1%, %, %, 3%, %, and 12% that were predicted by the disopred md, iupreds, pondr-fit, mfdp, and ucon, respectively; the native amount of disorder is %. the content-adjusted annotation of the disordered residues captures a large number of disordered residues in the long segment between positions and as well as the two disordered segments nearest to the c-terminus. these improvements come as a trade-off for an over-prediction of the disorder at the n-terminus, particularly for the pondr-fit and disopred predictors. overall, we observe that the adjusted predictions show a denser concentration of the disordered residues around the natively disordered regions. we conclude that although predictions shown in the two case studies should not be assumed typical, they demonstrate that the content predicted with discon offers valuable assistance in selection of the cut-offs to annotate the disordered residues based on the real-values predictions from modern disorder predictors. factors related to the amount of disorder/order we convert the input protein sequence into a custom-designed set of selected numerical descriptors which utilize information related to the evolutionary profiles, sequence itself, and predicted secondary structure, solvent accessible residues, b-factors and globular domains; see table a in the additional file majority of the selected features combine multiple input sources. for instance, the largest group of similar features is based on counting the residues in certain predicted ss states, with certain levels of predicted solvent exposure and b-factors which are located within the predicted domains. for instance, the sshedominbfnslowrsab feature counts the predicted helix and strand residues that are in globular domains, have low b-factors and are buried. as expected, these values are negatively correlated with the content of the native disorder, see figure since this descriptor highlights hallmarks of well-structured proteins, i.e., they usually include conserved domains with buried strands and/or helices that are usually structurally rigid. another feature that attains negative, -, correlation with the native content is bfnslowseg which quantifies the number of predicted rigid residues that are grouped together in the sequence in segments of size at least figure shows a cluster of bfnslowseg values between and for chains with low amount of native disorder, which suggests that well-structured proteins include significant amount of rigid residues that are grouped together in the sequence, while disordered chains contain fewer of such rigid residues. we also discuss two features that have relatively high positive correlation with the disorder content. the sschbfnshighdomnotin feature counts the number of coil and helix residues with high b-factor that are not in the globular domains. this feature that has pcc = in our dataset, see figure agrees with characteristic properties of the disorder, which often concerns flexible residues that are outside of the domains. the chc...chseg feature computes the number of residues in the longest segment in the predicted ss that does not include strands, i.e., stretch of the sequence that is composed only of a coils or of alternating helix-coil-helix-coil... segments. this feature demonstrates that disordered proteins are often depleted of beta-sheets. the input features also highlight the importance of the relation between sequence conservation and the amount of the disorder, i.e., out of the features utilize entropy computed from the pssm or wop profiles. for instance, the entavepssm feature, which computes the entropy of the average pssm scores for each column in the matrix along the sequence, has pcc = -. this means that well-structured proteins are characterized on average by a stronger degree of sequence conservation when compared with the disordered proteins. our observation is in agreement with the results of previous study, where the evolution rates of ordered and intrinsically disordered regions were compared using the pairwise genetic distances between the ordered and the disordered regions of protein families having at least one member with a structurally characterized region of disorder of or more consecutive residues. this study demonstrated that the disordered regions evolved significantly more rapidly than the ordered regions in of the families studied.  in spite of the fact that the quality of the high-throughput disorder prediction continues to improve, researchers recognize that new and more accurate predictors are still needed. we address the shortage of accurate methods that predict the overall amount of disorder in a given protein chain, which is motivated by the fact that current disorder predictors tend to provide relatively inaccurate estimates of the disorder content. we propose a novel approach, called discon, which combines information derived from sequence, sequence profiles, and predicted secondary structure, solvent accessibility, flexibility, and annotation of globular domains. we custom designed feature-based representation of the input protein chain that aggregates and combines these inputs and we performed feature selection that found a small set of complementary features that are well correlated with the native disorder. using these features and a ridge regression-based model, the discon predicts the disorder content with low,, mean squared error and high,, correlation, as evaluated on an independent test dataset. these predictions are empirically shown to be significantly better than the disorder content estimates derived from outputs of ten modern disorder predictors. the discon's predictions provide a high-quality alternative for high-throughput annotation of the disorder content. they are also shown to provide useful input to improve binary annotations of the disordered residues from the real-value disorder propensities generated by current disorder prediction methods. list of abbreviations auc: area under the roc curve; asa: absolute solvent accessibility; casp: critical assessment of techniques for protein structure prediction; cdf: cumulative distribution function; ch-plot: charge-hydropathy plot; discon: disorder content predictor; idp: intrinsically disordered protein; mae: mean absolute error; mcc: matthews correlation coefficient; mse: mean squared error; nmr: nuclear magnetic resonance; pcc: pearson correlation coefficient; pdb: protein data bank; pssm: position specific scoring matrix; puk: pearson vii function-based universal kernel; rbf: radial basis function; rsa: relative solvent accessibility; ss: secondary structure; svr: support vector regression; wop: weighted observed percentage. competing interests the authors declare that they have no competing interests. authors' contributions mjm contributed to the collection of the data, design of the prediction method, experimental validation, analysis and interpretation of the results, and implementation of the web server. tz worked on the design and implementation of the prediction method and analysis and interpretation of the results. bx helped with the analysis and interpretation of the results. yz and akd helped with the conception of the project and analysis and interpretation of the results. vnu contributed to the analysis and interpretation of the results and writing of the manuscript. lk was responsible for the conception of the project, and contributed to the collection of the data, design and implementation of the prediction method, experimental validation, analysis and interpretation of the results, and writing of the manuscript. all authors have read, corrected and approved the manuscript. supplementary material additional file 1 supplementary tables and figures. this file includes supplementary tables and supplementary figures. the tables summarize the input features and results obtained with alternative designs of the proposed predictor. the figures summarize correlation between the input features and the predictive target, the relation between the predictive quality and the input chain length, and the accuracy for the residue-level disorder predictions. click here for file acknowledgements this work was sponsored in part by the discovery grant from nserc canada to lk, the national institutes of health grant to yz, and the killam memorial scholarship to mjm. the funding agencies did not participate in the design, collection, analysis, and interpretation of the data.
we hypothesize that sequence-level aggregation of input information may provide more accurate content prediction when compared with the content extracted from the local window-based residue-level disorder predictors. the above investigations currently utilize the disorder content derived from the per-residue disorder predictions. we propose a novel predictor, discon, that takes advantage of a small set of custom-designed descriptors that aggregate and hybridize information concerning sequence, evolutionary profiles, and predicted secondary structure, solvent accessibility, flexibility, and annotation of globular domains. we also empirically demonstrate that the discon's predictions can be used to improve binary annotations of the disordered residues from the real-value disorder propensities generated by current residue-level disorder predictors. this is a statistically significant improvement over the content computed from outputs of ten modern disorder predictors on a test dataset with proteins that share low sequence identity with the training sequences. we show that these predictions may over-or under-predict the overall amount of disorder, which motivates development of novel tools for direct and accurate sequence-based prediction of the disorder content. the proposed predictive model is analyzed to discuss factors related to the prediction of the disorder content. discon is a high-quality alternative for high-throughput annotation of the disorder content. the knowledge of the content of the intrinsic disorder in proteins is useful for a variety of studies including estimation of the abundance of disorder in protein families, classes, and complete proteomes, and for the analysis of disorder-related protein functions. the web server that implements the discon is available at http://biomine.ece.ualberta.ca/discon/. using these descriptors and a ridge regression model, discon predicts the content with low,, mean squared error and high,, pearson correlation.
 intrinsically disordered proteins play important roles in various cellular activities and their prevalence was implicated in a number of human diseases. the knowledge of the content of the intrinsic disorder in proteins is useful for a variety of studies including estimation of the abundance of disorder in protein families, classes, and complete proteomes, and for the analysis of disorder-related protein functions. the above investigations currently utilize the disorder content derived from the per-residue disorder predictions. we show that these predictions may over-or under-predict the overall amount of disorder, which motivates development of novel tools for direct and accurate sequence-based prediction of the disorder content.  we hypothesize that sequence-level aggregation of input information may provide more accurate content prediction when compared with the content extracted from the local window-based residue-level disorder predictors. we propose a novel predictor, discon, that takes advantage of a small set of custom-designed descriptors that aggregate and hybridize information concerning sequence, evolutionary profiles, and predicted secondary structure, solvent accessibility, flexibility, and annotation of globular domains. using these descriptors and a ridge regression model, discon predicts the content with low,, mean squared error and high,, pearson correlation. this is a statistically significant improvement over the content computed from outputs of ten modern disorder predictors on a test dataset with proteins that share low sequence identity with the training sequences. the proposed predictive model is analyzed to discuss factors related to the prediction of the disorder content.  discon is a high-quality alternative for high-throughput annotation of the disorder content. we also empirically demonstrate that the discon's predictions can be used to improve binary annotations of the disordered residues from the real-value disorder propensities generated by current residue-level disorder predictors. the web server that implements the discon is available at http://biomine.ece.ualberta.ca/discon/. 