BACKGROUND
machine learning  tools have been successfully applied to the solution of a variety of biological problems such as the classification of proteins according to their subcellular localization and secretion mechanism.
different computational methods have been used to obtain reliable subcellular localization predictions, such as artificial neural networks , hidden markov models  and support vector machines   <cit> .
the simplest way of addressing classification problems is to follow a binary approach, trying to discriminate objects according to two categories: positive  and negative .
svms rely on two concepts in order to solve this type of problems: the first one is known as the large-margin separation principle, which is motivated by the idea of classifying points in two dimensions; and the second one is known as kernel methods  <cit> .
the kernel methods that have been applied to bioinformatics are classified into three categories mainly: kernels for real-valued data, kernels for sequences and kernels developed for specific purposes such as the position-specific scoring matrix -kernel  <cit> .
in the first case, examples that represent a data set can be usually expressed as feature vectors of a given dimensionality.
in the case of kernel functions for real-valued data, linear, polynomial and gaussian kernels are some of the most commonly used functions and they were used in the implementation of nclassg+.
in the third case, the most frequently used kernels for sequences are the spectrum kernels describing l-mer content  <cit> , positional weighted degree  kernels that use positional information  <cit>  and other kernels for sequences such as the local alignment kernel  <cit> .
the use of kernels for exploring real-valued biological data such as proteins usually involves two steps.
in the first step, amino acid sequences are transformed into fixed-length vectors that are then used to feed ml tools so that they can learn to make predictions in a second step  <cit> .
the svm classification method outstands among the techniques based on kernel learning, which searches for an optimal separation hyperplane in the feature space and determines the optimal data separation margin, maximizing the generalization capacity of the detected pattern.
this separation hyperplane is trained by means of quadratic programming  <cit> .
svms and kernel functions are very effective for solving classification problems because they are based on probability theory, can handle large data sets of high dimensionality, and have great flexibility to model diverse data sources  <cit> .
one of the fundamental issues of computational biology is directly associated with representing data as objects in a given space; this is of key importance for the solution of classification and clustering problems.
for example, in the case of protein sequences, their variable lengths do not allow the use of vector representations  <cit> , a problem known as the "sequence metric problem", which is directly associated with the use of an alphabetic letter code that lacks an implicit metric and, therefore, it is not suitable for making comparisons between such objects  <cit> .
to solve this problem, different sequence representations have been proposed based on features and similarity measures, some of which are shown in table  <dig> <cit> .
the split set is a partition of the learning data set used in the training process, and the test set corresponds to the independent set used in the final test only for comparing the performance of nclassg+, secretomep  <dig>  and secretp  <dig> .
the split sets  correspond to the ones reported in figure  <dig> .
over the last  <dig> years the use of the ml techniques mentioned above have allowed proposing novel solutions to the identification of protein secretion and post-translational modifications.
the validation of the different methods available for predicting protein secretion  <cit> , as well as the use of such algorithmic methods for the identification of potential drug and vaccine target proteins, followed by the experimental validation of such predictions  <cit> , have shown to be a consistent approach to obtain novel biological findings supported on computational processes and with direct application to the solution of protein secretion problems.
ml tools used in the identification of secreted proteins have been developed taking into account the biological principles of protein subcellular localization, which is essential for the correct functioning of these proteins  <cit> .
the localization of secreted proteins in their appropriate cellular compartments involves diverse processes that range from the transport of small molecules through highly complex routes with intrinsic sequence signaling processes.
much of the current efforts in understanding protein secretion have focused on how such protein transportation systems work and on the identification of membrane proteins to drive drug development toward products that have specific effects on such proteins  <cit> .
in gram-positive bacteria, proteins might localize in at least four different locations: the cytoplasm, cytoplasm membrane, cell wall and extracellular milieu.
since protein synthesis takes place in the cytoplasm, secreted proteins have to be transported across the cell membrane so that they can fulfill their function effectively  <cit> .
given the complexity of such secretion systems, it is not surprising that new mechanisms of secretion are being constantly discovered  <cit> .
thus, there is a considerable number of proteins that have been experimentally identified as secreted but whose mechanism or route of secretion has not been yet identified and therefore are said to be secreted via non-classical or alternative means  <cit> .
many of the proteins that are secreted via alternative pathways are directly associated with pathogenic processes, thus their identification is of key importance  <cit> .
in the case protein secretion in gram-positive bacteria, there are six secretion systems to transport proteins across the cytoplasmic membrane reported up to date: secretion , twin-arginine translocation , flagella export apparatus , fimbrilin-protein exporter , hole forming  and wxg <dig> secretion system  <cit> ; however, it is important to emphasize that non-classical protein secretion should not be considered as a single mechanism but rather as a range of secretion systems that differ from classical secretion but are still not clearly characterized.
this discloses problems both with the experimental and computational strategies currently used to identify new secretory mechanisms and highlights the importance of developing new strategies to study non-classical secretion.
the development of this work focused on the identification of non-classically secreted proteins.
it is worth noting that for some of these secreted proteins a known function has been also reported in the cytoplasm, leading to their classification as "moon-lightning" or multi-functional proteins.
nclassg+ identifies proteins that are secreted through signal-peptide independent pathways and was here validated based on a compiled list of extracellular proteins lacking a signal peptide.
nclassg+ was compared to the two available algorithms for classifying non-classically secreted gram-positive proteins, named secretomep  <dig>   <cit>  and secretp  <dig>   <cit> .
RESULTS
a training and a split set were built from a learning data set containing  <dig> positive proteins and  <dig> negative proteins with thoroughly adjusted parameters.
independently, a test set containing  <dig> positive examples of non-classically secreted proteins and  <dig> negative examples were constructed for comparing nclassg+ to the other classifiers of non-classical secretion.
these data sets were the result of removing redundant proteins with more than 25% of identity.
linear, polynomial and gaussian kernel functions were selected for constructing the representation vectors, as literature revision indicated that these are very well explored kernel functions.
the data sets were supported on experimental reports and the necessary vector transformations were applied to them during the learning process.
a nested k-fold cv procedure was used to tune the model and compute the error separately.
this was done with the aim of finding the best parameters to train the complete data set.
the exploration was optimized using a grid search approach and led to proposing a classifier, which was trained independently on frequencies, dipeptides, factors and pssm vectors as well as on all possible combinations between such vectors.
the predictive behavior of nclassg+ was analyzed and contrasted against secretomep  <dig>  and secretp  <dig>  in two occasions: one with the split set during the training process and the other one with the test set during a separate testing step.
model selection
about  <dig>  <dig> hyperparameter combinations comprising feature vectors, svm c values, and kernel functions and their parameters were explored to select the best classifier.
the optimized exploration of combinations pointed to a linear classifier combining factors, dipeptides and pssm vectors as the one that yielded the highest accuracy in the inner loop of the nested cv procedure.
the c parameter of the classifier was equal to  <dig>  the average accuracy of the outer folds in the nested k-fold cross-validation was  <dig> .
evaluation measurements
in figure  <dig> the roc plot of nclassg+ shows the true positive rate  plotted in function of the false positive rate .
the roc plot test shows good discrimination.
the graph also shows a high accuracy as the curve climbs rapidly toward the upper left hand corner of the graph.
compared to secretomep and secretp, nclassg+ showed a better performance both in the test with the split set after the training process, as well as in the independent test with the test set, as indicated by its higher accuracy and mcc.
the correct identification of non-classically secreted and non-secreted proteins, understood in terms of the tools' sensitivity and specificity, were notably high for nclassg+ , thus indicating that this tool recognizes a similar proportion of both protein types, in contrast to secretomep  <dig>  and secretp  <dig> , in which such relationships were unbalanced .
discussion
one of the most complex areas of ml is directly associated with finding and constructing training and exploration data sets  <cit> .
in this study, a positive training set containing  <dig>  <dig> protein sequences and a negative training set comprising  <dig>  <dig> protein sequences were obtained by screening the swissprot database.
both protein sets were balanced by adjusting the percentage of identity in each set.
in this study, prediction of non-classically secreted proteins is done based on a modification of classically secreted proteins, as proposed by bendtsen and colleagues  <cit> .
however, here we postulate novel training and exploration data sets that were astringently adjusted, as well as innovative data transformations and methods not previously used in the classification of non-classically secreted proteins.
it is important to highlight that the input data for the construction of nclassg+, secretomep  <dig>  and secretp  <dig>  were all extracted from swissprot ; therefore, there is probably some data overlapping between the training data sets of the three tools.
nevertheless, the diversity of protein prediction methods, the constant increase of protein data and the identification of new problems stress the importance of analyzing and extracting data to construct new hypotheses in terms of protein localization.
different pre-processing techniques were used in the construction of the feature vectors that represented each of the sequences in the input data set.
these techniques have some intrinsic computation details that can result in comparatively more expressive vectors  <cit> .
in the specific case of dipeptide and pssm vectors, both types of vectors use  <dig> features to represent each amino acid sequence, but evidently, pssm is the vector that represents each protein more effectively.
pssm vectors have been reported to be one of the most efficient ways of representing proteins in statistical learning  <cit>  but the strategy of mixing different vectors resulted in even better results in terms of the evaluation measurements.
it is worth noting that nclassg+, secretomep  <dig>  and secretp  <dig>  use data from two biological classes of gram-positive bacteria .
however, part of the features used in secretomep  <dig>  come from prediction methods that were trained with protein sequences that belong to biological groups different from gram-positive bacteria, which suggests that there are common secretion mechanisms among the different biological entities; however, such hypothesis should be experimentally validated in the same way as it has been done for classical secretion in gram-positive bacteria  <cit> .
although both nclassg+ and secretp  <dig>  use an svm algorithm, there are deep differences in terms of the methodology approach followed by both tools.
both tools use different techniques to build their vector representations, but secretp  <dig>  does a smaller exploration to obtain its final classifier.
yu et al.
reported a lower ability of secretp  <dig>  to predict non-classically secreted gram-positive proteins compared to secretomep  <dig>   <cit> , which also agrees with the results of nclassg+ .
however, it is particularly interesting that secretp  <dig>  was built to classify  <dig> protein categories  but was validated using classical measures , which are basically adequate to evaluate binary results.
in particular for nclassg+, the linear, polynomial and gaussian kernel functions were explored under equal conditions for its optimization.
the best results were obtained using the linear function, which is consistent with reports by ben-hur and colleagues  <cit>  stating that the linear kernel provides a useful baseline and is hardly beaten in many bioinformatics applications, especially when the dimensionality of the input set is large and there is a small number of samples, as occurred with nclassg+.
in order to select the best classifier, the results were optimized according to parameters, exploring different vector combinations as well as different kernel functions.
in the case of the function exploration, it is important to mention that the gaussian function has less difficulties compared to the polynomial function because  <dig> <kij ≤  <dig>  in contrast to the polynomial kernel function, where values may tend to infinity as the degree of the polynomial increases  <cit> .
this is observed in the nature of the variables of the polynomial function, where the number of experiments is larger compared to the other two methods .
in the validation of the different classifiers proposed in this study, the results obtained by calculating the roc showed good discrimination between false positives and true positive proteins.
nevertheless, it should be taken into account that the rocs characterize the potential ranges of the algorithm but not the performance of a given classifier  <cit> .
CONCLUSIONS
this study reports the nclassg+ tool for the classification of gram-positive bacterial proteins that are secreted independently of the classical secretory pathway.
this tool has a novel training data set and is composed of a classifier based on a polynomial function that uses vectors built from dipeptides, frequencies and pssm data.
among the  <dig> types of vectors, the similarity-based pssm vector was always present in the optimization process, which reflects the efficiency of this type of vector for representing protein sequences, compared to the other  <dig> types of vectors.
however, the combination of the different vector representations was a good approach to solve the classification problem, as it minimized the optimistic biased thanks to the nested cv and allowed to obtain a robust classifier.
there are still novel protein secretion and translocation mechanisms to be discovered, where the use of computational and ml methods can play a key role for elucidating new processes and discovering new biological mechanisms.

 machine learning tools have been successfully applied to the solution of a variety of biological problems such as the classification of proteins according to their subcellular localization and secretion mechanism. different computational methods have been used to obtain reliable subcellular localization predictions, such as artificial neural networks, hidden markov models and support vector machines. the simplest way of addressing classification problems is to follow a binary approach, trying to discriminate objects according to two categories: positive and negative. svms rely on two concepts in order to solve this type of problems: the first one is known as the large-margin separation principle, which is motivated by the idea of classifying points in two dimensions; and the second one is known as kernel methods. the kernel methods that have been applied to bioinformatics are classified into three categories mainly: kernels for real-valued data, kernels for sequences and kernels developed for specific purposes such as the position-specific scoring matrix -kernel. in the first case, examples that represent a data set can be usually expressed as feature vectors of a given dimensionality. in the case of kernel functions for real-valued data, linear, polynomial and gaussian kernels are some of the most commonly used functions and they were used in the implementation of nclassg+. in the third case, the most frequently used kernels for sequences are the spectrum kernels describing l-mer content, positional weighted degree kernels that use positional information and other kernels for sequences such as the local alignment kernel. the use of kernels for exploring real-valued biological data such as proteins usually involves two steps. in the first step, amino acid sequences are transformed into fixed-length vectors that are then used to feed ml tools so that they can learn to make predictions in a second step. the svm classification method outstands among the techniques based on kernel learning, which searches for an optimal separation hyperplane in the feature space and determines the optimal data separation margin, maximizing the generalization capacity of the detected pattern. this separation hyperplane is trained by means of quadratic programming. svms and kernel functions are very effective for solving classification problems because they are based on probability theory, can handle large data sets of high dimensionality, and have great flexibility to model diverse data sources. one of the fundamental issues of computational biology is directly associated with representing data as objects in a given space; this is of key importance for the solution of classification and clustering problems. for example, in the case of protein sequences, their variable lengths do not allow the use of vector representations, a problem known as the "sequence metric problem", which is directly associated with the use of an alphabetic letter code that lacks an implicit metric and, therefore, it is not suitable for making comparisons between such objects. to solve this problem, different sequence representations have been proposed based on features and similarity measures, some of which are shown in table. the split set is a partition of the learning data set used in the training process, and the test set corresponds to the independent set used in the final test only for comparing the performance of nclassg+, secretomep and secretp. the split sets correspond to the ones reported in figure. over the last years the use of the ml techniques mentioned above have allowed proposing novel solutions to the identification of protein secretion and post-translational modifications. the validation of the different methods available for predicting protein secretion, as well as the use of such algorithmic methods for the identification of potential drug and vaccine target proteins, followed by the experimental validation of such predictions, have shown to be a consistent approach to obtain novel biological findings supported on computational processes and with direct application to the solution of protein secretion problems. ml tools used in the identification of secreted proteins have been developed taking into account the biological principles of protein subcellular localization, which is essential for the correct functioning of these proteins. the localization of secreted proteins in their appropriate cellular compartments involves diverse processes that range from the transport of small molecules through highly complex routes with intrinsic sequence signaling processes. much of the current efforts in understanding protein secretion have focused on how such protein transportation systems work and on the identification of membrane proteins to drive drug development toward products that have specific effects on such proteins. in gram-positive bacteria, proteins might localize in at least four different locations: the cytoplasm, cytoplasm membrane, cell wall and extracellular milieu. since protein synthesis takes place in the cytoplasm, secreted proteins have to be transported across the cell membrane so that they can fulfill their function effectively. given the complexity of such secretion systems, it is not surprising that new mechanisms of secretion are being constantly discovered. thus, there is a considerable number of proteins that have been experimentally identified as secreted but whose mechanism or route of secretion has not been yet identified and therefore are said to be secreted via non-classical or alternative means. many of the proteins that are secreted via alternative pathways are directly associated with pathogenic processes, thus their identification is of key importance. in the case protein secretion in gram-positive bacteria, there are six secretion systems to transport proteins across the cytoplasmic membrane reported up to date: secretion, twin-arginine translocation, flagella export apparatus, fimbrilin-protein exporter, hole forming and wxg secretion system; however, it is important to emphasize that non-classical protein secretion should not be considered as a single mechanism but rather as a range of secretion systems that differ from classical secretion but are still not clearly characterized. this discloses problems both with the experimental and computational strategies currently used to identify new secretory mechanisms and highlights the importance of developing new strategies to study non-classical secretion. the development of this work focused on the identification of non-classically secreted proteins. it is worth noting that for some of these secreted proteins a known function has been also reported in the cytoplasm, leading to their classification as "moon-lightning" or multi-functional proteins. nclassg+ identifies proteins that are secreted through signal-peptide independent pathways and was here validated based on a compiled list of extracellular proteins lacking a signal peptide. nclassg+ was compared to the two available algorithms for classifying non-classically secreted gram-positive proteins, named secretomep and secretp.  a training and a split set were built from a learning data set containing positive proteins and negative proteins with thoroughly adjusted parameters. independently, a test set containing positive examples of non-classically secreted proteins and negative examples were constructed for comparing nclassg+ to the other classifiers of non-classical secretion. these data sets were the result of removing redundant proteins with more than 25% of identity. linear, polynomial and gaussian kernel functions were selected for constructing the representation vectors, as literature revision indicated that these are very well explored kernel functions. the data sets were supported on experimental reports and the necessary vector transformations were applied to them during the learning process. a nested k-fold cv procedure was used to tune the model and compute the error separately. this was done with the aim of finding the best parameters to train the complete data set. the exploration was optimized using a grid search approach and led to proposing a classifier, which was trained independently on frequencies, dipeptides, factors and pssm vectors as well as on all possible combinations between such vectors. the predictive behavior of nclassg+ was analyzed and contrasted against secretomep and secretp in two occasions: one with the split set during the training process and the other one with the test set during a separate testing step. model selection about hyperparameter combinations comprising feature vectors, svm c values, and kernel functions and their parameters were explored to select the best classifier. the optimized exploration of combinations pointed to a linear classifier combining factors, dipeptides and pssm vectors as the one that yielded the highest accuracy in the inner loop of the nested cv procedure. the c parameter of the classifier was equal to the average accuracy of the outer folds in the nested k-fold cross-validation was. evaluation measurements in figure the roc plot of nclassg+ shows the true positive rate plotted in function of the false positive rate. the roc plot test shows good discrimination. the graph also shows a high accuracy as the curve climbs rapidly toward the upper left hand corner of the graph. compared to secretomep and secretp, nclassg+ showed a better performance both in the test with the split set after the training process, as well as in the independent test with the test set, as indicated by its higher accuracy and mcc. the correct identification of non-classically secreted and non-secreted proteins, understood in terms of the tools' sensitivity and specificity, were notably high for nclassg+, thus indicating that this tool recognizes a similar proportion of both protein types, in contrast to secretomep and secretp, in which such relationships were unbalanced. discussion one of the most complex areas of ml is directly associated with finding and constructing training and exploration data sets. in this study, a positive training set containing protein sequences and a negative training set comprising protein sequences were obtained by screening the swissprot database. both protein sets were balanced by adjusting the percentage of identity in each set. in this study, prediction of non-classically secreted proteins is done based on a modification of classically secreted proteins, as proposed by bendtsen and colleagues. however, here we postulate novel training and exploration data sets that were astringently adjusted, as well as innovative data transformations and methods not previously used in the classification of non-classically secreted proteins. it is important to highlight that the input data for the construction of nclassg+, secretomep and secretp were all extracted from swissprot; therefore, there is probably some data overlapping between the training data sets of the three tools. nevertheless, the diversity of protein prediction methods, the constant increase of protein data and the identification of new problems stress the importance of analyzing and extracting data to construct new hypotheses in terms of protein localization. different pre-processing techniques were used in the construction of the feature vectors that represented each of the sequences in the input data set. these techniques have some intrinsic computation details that can result in comparatively more expressive vectors. in the specific case of dipeptide and pssm vectors, both types of vectors use features to represent each amino acid sequence, but evidently, pssm is the vector that represents each protein more effectively. pssm vectors have been reported to be one of the most efficient ways of representing proteins in statistical learning but the strategy of mixing different vectors resulted in even better results in terms of the evaluation measurements. it is worth noting that nclassg+, secretomep and secretp use data from two biological classes of gram-positive bacteria. however, part of the features used in secretomep come from prediction methods that were trained with protein sequences that belong to biological groups different from gram-positive bacteria, which suggests that there are common secretion mechanisms among the different biological entities; however, such hypothesis should be experimentally validated in the same way as it has been done for classical secretion in gram-positive bacteria. although both nclassg+ and secretp use an svm algorithm, there are deep differences in terms of the methodology approach followed by both tools. both tools use different techniques to build their vector representations, but secretp does a smaller exploration to obtain its final classifier. yu et al. reported a lower ability of secretp to predict non-classically secreted gram-positive proteins compared to secretomep, which also agrees with the results of nclassg+. however, it is particularly interesting that secretp was built to classify protein categories but was validated using classical measures, which are basically adequate to evaluate binary results. in particular for nclassg+, the linear, polynomial and gaussian kernel functions were explored under equal conditions for its optimization. the best results were obtained using the linear function, which is consistent with reports by ben-hur and colleagues stating that the linear kernel provides a useful baseline and is hardly beaten in many bioinformatics applications, especially when the dimensionality of the input set is large and there is a small number of samples, as occurred with nclassg+. in order to select the best classifier, the results were optimized according to parameters, exploring different vector combinations as well as different kernel functions. in the case of the function exploration, it is important to mention that the gaussian function has less difficulties compared to the polynomial function because <kij ≤ in contrast to the polynomial kernel function, where values may tend to infinity as the degree of the polynomial increases. this is observed in the nature of the variables of the polynomial function, where the number of experiments is larger compared to the other two methods. in the validation of the different classifiers proposed in this study, the results obtained by calculating the roc showed good discrimination between false positives and true positive proteins. nevertheless, it should be taken into account that the rocs characterize the potential ranges of the algorithm but not the performance of a given classifier.  this study reports the nclassg+ tool for the classification of gram-positive bacterial proteins that are secreted independently of the classical secretory pathway. this tool has a novel training data set and is composed of a classifier based on a polynomial function that uses vectors built from dipeptides, frequencies and pssm data. among the types of vectors, the similarity-based pssm vector was always present in the optimization process, which reflects the efficiency of this type of vector for representing protein sequences, compared to the other types of vectors. however, the combination of the different vector representations was a good approach to solve the classification problem, as it minimized the optimistic biased thanks to the nested cv and allowed to obtain a robust classifier. there are still novel protein secretion and translocation mechanisms to be discovered, where the use of computational and ml methods can play a key role for elucidating new processes and discovering new biological mechanisms. 
this study describes the implementation of a sequence-based classifier, denoted as nclassg+, for identifying non-classically secreted gram-positive bacterial proteins. several feature-based classifiers were trained using different sequence transformation vectors and support vector machines with linear, polynomial and gaussian kernel functions. the parameters and kernel functions and the combinations between all possible feature vectors were optimized using grid search. nested k-fold cross-validation was applied to select the best models, using the inner cv loop to tune the model parameters and the outer cv group to compute the error. in fact, only two methods have been reported for predicting non-classically secreted proteins of gram-positive bacteria. the final model was tested against an independent set not previously seen by the model, obtaining better predictive performance compared to secretomep v and secretpv for the identification of non-classically secreted proteins.
 most predictive methods currently available for the identification of protein secretion mechanisms have focused on classically secreted proteins. in fact, only two methods have been reported for predicting non-classically secreted proteins of gram-positive bacteria. this study describes the implementation of a sequence-based classifier, denoted as nclassg+, for identifying non-classically secreted gram-positive bacterial proteins.  several feature-based classifiers were trained using different sequence transformation vectors and support vector machines with linear, polynomial and gaussian kernel functions. nested k-fold cross-validation was applied to select the best models, using the inner cv loop to tune the model parameters and the outer cv group to compute the error. the parameters and kernel functions and the combinations between all possible feature vectors were optimized using grid search.  the final model was tested against an independent set not previously seen by the model, obtaining better predictive performance compared to secretomep v and secretpv for the identification of non-classically secreted proteins. nclassg+ is freely available on the web at http://www.biolisi.unal.edu.co/web-servers/nclassgpositive/ 