BACKGROUND
in eukaryotes, the majority of proteins are encoded in the nuclear genome and translated on ribosomes in the cytosol.
proteins are then transported to different subcellular locations, such as the nucleus, mitochondria, chloroplasts, peroxisomes, etc., where they perform their particular roles in various biological processes.
knowledge of subcellular location is an important asset in the annotation of newly discovered proteins, as it bears clues about a protein's function.
further, knowing the location of proteins and their molecular function allows us to infer where in the cell the corresponding biological process takes place, what the physiological role of this process may be, and how the various processes are spatially integrated.
finally, information on the makeup of proteomes from bacteria-derived organelles  helps to elucidate the migration of protein-coding genes from the endosymbiont to the host.
a variety of experimental approaches are available today for identifying the subcellular localization of proteins, for example, co-expression of fluorescent proteins  <cit> , immunofluorescence labeling  <cit> , gene knockout/knockdown  <cit> , and proteomics techniques such as liquid-chromatography-tandem mass spectrometry   <cit> .
however, for most species, large-scale experimental identification of protein subcellular localization remains too expensive or unfeasible.
this has set the stage for bioinformatics approaches to predict localization in silico.
can localization of a protein be confidently inferred via finding a homolog of known location by blast  <cit> ?
a previous study indicated that localization can be predicted with up to 90% accuracy when blast identity is 50% or more, but that it falls short for more distant sequences   <cit> .
further, this approach ignores established biological knowledge that homologous proteins are not necessarily located in the same cellular compartment.
for example, homologous beta oxidation enzymes are targeted to mitochondria in human and peroxisomes in yeast  <cit> .
most importantly, the blast approach fails for divergent and novel proteins as they do not find significant matches in databases .
for all these reasons, the bioinformatics community turned to more suited approaches for protein localization prediction.
today, more than  <dig> dedicated tools are available for in silico protein localization prediction based on annotation or solely the sequence of proteins .
annotation information includes textual description taken from the swissprot database, the gene ontology database, or pubmed literature  <cit> .
also used for localization prediction is co-occurrence of functional motifs or structural domains in proteins  <cit> .
sequence-based tools recognize specific targeting signals that guide proteins to different cellular compartments  <cit> .
alternatively, proteins are classified according to single amino acid frequency  <cit> , dipeptide and gapped amino acid pair composition  <cit> , or physicochemical properties of amino acids  <cit> .
more recently published predictors combine different protein features  <cit> , or integrate annotation with sequences-based prediction  <cit> .
finally, meta-predictors combine predictions from several heterogeneous tools  <cit> .
two recent studies evaluated the performance of available localization predictors using datasets that contain only sequences not included in, nor similar to, those in the training sets of these predictors  <cit> .
one identified as best performing tools bacello  <cit> , loctree  <cit> , protein prowler  <cit> , targetp  <cit> , and wolf-psort  <cit> , and the other evaluated bacello, yloc  <cit> , muitiloc <dig>  <cit> , and knowpred  <cit>  as best .
in general, these tools have lower performance on data from plants compared to non-photosynthetic organisms such as animals and fungi, and this is due to the presence of mitochondria plus chloroplasts in the cell of plants.
both organelles descend from endosymbiotic bacteria and have their own machineries for protein import, dna replication, and gene expression.
this makes it difficult for the tools to distinguish the proteins from the two organelles.
in silico localization prediction tools use full-length protein sequences that are usually inferred from genome sequence.
yet, for many eukaryotic groups of interest are only est  data available, and it is unlikely that their genomes will be sequenced soon  <cit> .
database  <cit> , and the taxonomically broad est database   <cit> ).
when attempting to use available localization prediction tools on protein sequences conceptually translated from ests, we realized that prediction accuracy is generally very low.
we tested the performance of seven state-of-the-art tools with proteins inferred from plant ests, and the overall accuracies were below 50% .
this is not surprising, because these tools have been designed for full-length proteins and not for ests, which often represent only partial coding regions with an average length of ~ <dig> residues.
further, est-inferred proteins  may have an amino acid composition that differs from that of the corresponding full-length proteins.
more importantly, est-peptides often lack the n-terminal region of the corresponding proteins, which usually contains the targeting signal.
<dig> numbers in %.
bold numbers are the result of the here described testloc, which is tailored for ests.
the results of full-length proteins are compiled in additional file 8
<dig> abbreviations: chl, chloroplast; cyt, cytosol; end, endoplasmatic reticulum; ext, extracellular space; mit, mitochondrion; nuc, nucleus; per, peroxisome; pla, plasma membrane; vac, vacuole; sn, sensitivity; ppv, positive predictive value
<dig> note that all test data have homologs in databases, which in practice is rarely the case; see text
finally blast, which we showed above to be unsuited for localization prediction of full-length proteins, is equally unsuited on est-peptide data.
even at sequence identity levels above 90%, the class-averaged accuracy for plant ests was below 75% .
in practice, the accuracy would be even lower as est projects often discover novel proteins that lack matches in databases.
for example, in a large-scale protist est project, more than 60% of ests could not find informative matches  <cit> .
for the ests from such projects, the overall accuracy of localization prediction by blast would thus be less than 30%.
we set out to develop a method that is tailored for predicting subcellular localization based on ests.
as a test case we used plant data, which, as mentioned above, are more challenging than those from non-photosynthetic taxa.
the methodology we developed can be readily applied to ests from any taxonomic groups, and the models we constructed can be easily retrained with sequences from a particular taxon of interest.
methods
datasets
we used two datasets: data from all plants to build and evaluate the localization prediction models , and arabidopsis-only data to evaluate the combined prediction of orfs and localization .
collection of protein sequences of experimentally verified subcellular location
from swissprot, we collected full-length arabidopsis proteins localized in nine subcellular compartments: chloroplasts , cytosol , endoplasmic reticulum , extracellular space , mitochondria , nucleus , peroxisomes , plasma membrane , and vacuole .
protein sequences were selected by the following criteria: 1) they are encoded by the nuclear genome; 2) their subcellular localization is experimentally verified; and 3) the localization annotation is unambiguous .
these are strict criteria in order to avoid false positives in the dataset.
no bias was detected as to the functional categories of proteins collected this way compared to other studies  <cit>  .
arabidopsis ests dataset
the ests corresponding to the above arabidopsis proteins were found via a similarity search by blastx in dbest of genbank, using a procedure illustrated in additional file  <dig>  when the aligned region of an est was longer than 90% of its total length and the amino acid identity between the protein and the translated est was >90%, we regarded the pair of est and protein as belonging to the same gene.
the selected ests were translated by emboss transeq  <cit>  into amino acid sequences in the frame indicated by the blastx alignment.
sequence redundancy within the collected data was reduced by the tool cd-hit  <cit>  so that no pair of sequences shares more than 60% identity.
we obtained a dataset of  <dig> ests.
table  <dig> compiles the number of instances in each subcellular class.
plant ests dataset
using the same procedure as described above for arabidopsis, we generated est-peptides for all other plants with known localization .
these combined with the above described arabidopsis dataset constitute the plant dataset , which was then used to build the expanded plant dataset as specified in the following.
expanded plant ests dataset
machine learning schemes perform better when trained with larger datasets.
but in practice, the size of training data is often limited by their availability.
we noticed that in our initial collection of est-peptides, a number of proteins with known subcellular location were absent, because they have no corresponding est sequences in public databases.
to construct a training set with optimal coverage, the missing est-peptides were substituted by artificial ones, generated by breaking up full-length proteins into overlapping pieces of ~ <dig> residues.
in this way, we almost tripled the size of training data.
the procedure is described below.
the collected full-length protein sequences from plants were processed according to the following rules :
when a sequence was shorter than  <dig> residues, it remained unchanged.
when a sequence was  <dig> to  <dig> residues long, fragments of length ranging from  <dig> to  <dig> residues were generated from both the n-terminus and c-terminus.
the range was based on a survey of the length distribution of ests, which revealed a mean of ~ <dig> nt with a standard deviation of ~ <dig> nt.
the n-terminal fragment started within  <dig> residues from the first methionine, and the c-terminal fragment ended at the last amino acid.
this simulated the nature of ests, which usually contain the complete c-terminal, but lack the n-terminal region.
when a sequence was longer than  <dig> residues, an additional central fragment was generated starting anywhere downstream of the first  <dig> residues, but before the middle position of the original sequence.
the fragmented protein sequences were combined with est-derived peptides and clustered by cd-hit  using a threshold of 60%.
the final dataset contains sequences from  <dig> species.
localization prediction by dedicated tools and blast
we collected the best-performing localization prediction tools: targetp, protein prowler, bacello, wolf-psort, yloc, knowpred, and multiloc <dig> , and tested their performance on the expanded plant dataset.
the results of wolf-posrt and multiloc <dig> were obtained from a locally installed version.
the prediction of yloc was provided by its author.
for the remaining tools, the results were obtained from their corresponding web-server.
to assess how blast performs for localization prediction of ests, the est data were blasted against proteins in swissprot, and the localization information of the second match  is transferred to the ests.
implementation of support vector machine
features used to represent the peptide sequences for input of svm
physicochemical properties
physicochemical properties of amino acids in a protein can be represented by amino acid indices ).
the database currently contains  <dig> features for each amino acid .
for each amino acid feature, its value was added up for the whole sequence, and was normalized by the sequence length.
subsequently, each est-peptide was converted into a 494-dimension vector.
amino acid composition
six different types of amino acid composition were calculated.
these include the frequency of individual amino acids , di-peptides , tri-peptides , tetra-peptides , penta-peptides , and hexa-peptides  in the input sequence.
grouped amino acid composition
amino acids were grouped according to their properties .
the alphabet of  <dig> amino acids was replaced by an alphabet of size eight  or size ten .
group c classified amino acids according to their chemical properties, which have shown good performance for localization prediction of full-length proteins  <cit> .
group d classified amino acids according to their structure  <cit> .
after converting est sequences using these new alphabets, the composition of amino acid groups was calculated from 1st to 8th order.
gapped amino acid composition
this feature represents the frequency of two amino acids  separated by x residues, x being the gap length.
we experimented with gap lengths from  <dig> to  <dig>
parameter selection and evaluation of svm predictions
in this study, we employed the svm package libsvm  <cit> , with the radial basis function  adopted as kernel function  = exp, γ>0), which requires the selection of the kernel parameter γ, and the penalty parameter c. to obtain the optimal parameters and to evaluate the predictions, we performed a 10-fold cross validation scheme for svm parameter  selection, followed by a 10-fold independent evaluation to assess the prediction performance .
we first randomly divided the whole dataset into ten subsets of equal size.
for each iteration of the ten rounds, nine subsets were combined to build svm models, and the remaining subset was used for evaluation.
the combined nine subsets were further subdivided into ten groups, whereof nine groups were combined and used to train svm with given values for c and γ, while the remaining group was used to find the optimal combination of the two parameters.
finally, we assessed the performance of the selected c and γ values by using the evaluation data subset.
performance evaluation
we calculated the overall accuracy for all classes combined, as well as the sensitivity , specificity , positive predictive value , and matthews correlation coefficient  of each individual class as follows:
overall accuracy =∑i=1ntpi∑i=1n* <dig>
i: the i-th class; n: total number of classes
for each class i:
sensitivity =tpitpi+fni* <dig>
specificity =tnitni+fpi* <dig>
positive predictive value =tpitpi+fpi* <dig>
matthews correlation coefficient =tpi*tni−fpi*fni***
tp: true positives; fp: false positives; tn: true negatives; fn: false negatives
open reading frame  prediction for ests
prot4est  <cit>  was used for the prediction of open reading frames  in est sequences.
ests were first aligned with proteins from the ncbi non-redundant sequence database by blastx.
the protein-est alignment indicates the correct translation frame.
for ests without significant blast matches, we used estscan  <cit>  from within prot4est.
estscan predicts orfs based on a hidden markov model  by recognizing the species-specific bias in hexanucleotide composition associated with coding and non-coding regions, and generates a matrix to represent this bias  <cit> .
to generate the matrix, we trained estscan with all annotated arabidopsis genomic and mrna data collected from the european molecular biology laboratory  nucleotide sequence database  <cit> .
the mrna data were mapped to genomic data in order to find the borders of coding/non-coding regions, which was needed to train the hmm.
more details can be found in the publication describing est-scan  <cit> .
RESULTS
we experimented with different sequence features to represent est-peptides in svm-based prediction of subcellular localization.
as detailed in the methods section, features included amino acid composition, grouped amino acid composition reflecting the physicochemical properties, gapped amino acid composition capturing the spatial context, as well as combinations thereof.
the performance of the obtained prediction schemes varies considerably as shown below.
performance of predictors based on individual features
among the  <dig> sequence features investigated , the best performance was obtained by the svm predictor exploiting the 4th order amino acid composition , with a mcc > <dig>  for all large classes .
a similarly good performance was observed with svm predictors based on the 6th order group-c amino acid composition and the 7th order group-d amino acid composition .
unexpectedly, sequence features such as gapped amino acid composition and physicochemical properties represented by aaindices did not yield satisfying results, with mccs for most classes below  <dig>  .
<dig> numbers are the average of the 10-fold test.
numbers in parenthesis are the standard deviation.
bold numbers indicate the best values for each metric  in each class of the expanded plant dataset.
the values for sn and ppv are given in %.
mcc, matthews correlation coefficient.
for other abbreviations, see footnote to table  <dig>
<dig> the three features are 4th order amino acid composition, 6th order group-c amino acid composition and 7th order group-d amino acid composition
three classes  are underrepresented, because data from these locations are still scarce.
we included these classes because in practice, a query sequence could be from any of these locations.
were these locations left out, the predictor would inevitably misassign query sequences from these three classes.
yet, when these locations are included in the predictor, there is at least the possibility that the query sequence will be correctly assigned, even if the prediction accuracy may be low .
eventually, this shortcoming will be alleviated when more data from these locations become available.
performance of predictors based on combined features
previous studies showed that integration of multiple sequence features improves the performance of localization prediction  <cit> .
we combined all the  <dig> sequence features described in the methods section first by integrating attributes and second by integrating prediction results.
to integrate attributes, the vectors of all sequence features were combined and used as input for svm predictors.
this type of integration yielded much lower performance than the best predictor based on a single feature .
integration of prediction results from individual features was achieved by a two-layer svm .
the first layer consisted of svm predictors based on a single sequence feature, yielding as output the probabilities of the query sequence to belong to each class.
the outputs of all first-layer svms were combined and served as input for the second-layer svm.
thus, each sequence was converted to a vector of size  <dig> .
the two-layer svm predictor showed similarly good performance as that based on the best single feature .
in addition, we integrated the three features that yielded best performance , as well as the predictions from these features.
again, these schemes showed a similar mcc as the predictors based on a single feature, but with lower sensitivity and higher positive predictive value for most classes .
compared with the prediction from each of the three features individually, their integration often produced a much smaller rate of false positive predictions.
orf prediction from ests
another challenge in est-peptide-based localization prediction is the correct translation from nucleotide sequence.
unlike genomic data, ests often lack start codon and 5'-utr, which otherwise help to detect the correct open reading frame .
in addition, ests are products of single-pass reads often containing low quality regions with sequencing errors that further complicate the task.
several tools have been developed for orf identification in est sequences  <cit> .
we chose prot4est  <cit> , which combines similarity-based and machine-learning-based prediction of orfs.
the scoring matrix specific to arabidopsis was obtained by training estscan with arabidopsis data, as described in the methods section.
the accuracy of orf prediction, evaluated by the percentage of correctly identified start/stop positions of coding regions, was over 70% .
implementation of prediction methods and validation with arabidopsis data
we built a tool named testloc that combines est translation with localization prediction.
testloc has two components.
the first takes est nucleotide sequences as input and generates est-peptides via the tool prot4est.
the second takes est-peptides as input and outputs the probability that the peptide is targeted to a given subcellular compartment.
the current model predicts nine locations: cytosol, endoplasmic reticulum, extracellular space, mitochondria, nucleus, peroxisomes, plasma membrane, vacuole, and chloroplasts.
to evaluate the combined prediction of orfs and subcellular localization, we tested the performance of testloc with arabidopsis ests that correspond to proteins of known localization.
sequences were represented by three best-performing features: 4th order amino acid composition, 6th order group-c amino acid composition, and 7th order group-d amino acid composition.
the resulting predictions showed high mcc values  for most classes .
testloc can be downloaded and executed locally.
the sequence features to use and how to combine them can be chosen by users via editing the configuration file.
it should be emphasized that testloc allows users to train new models with their own data, which facilitates the analysis of sequences from other taxonomic groups or individual species.
note that testloc is designed for est data, and should not be applied to full-length proteins .
discussion
effects of various sequence features
when experimenting with different kinds of nth order amino acid composition, we observed a common trend: for the training data, the performance improved with increasing order until reaching a peak; for the evaluation data, at first the performance improved with the order, reached a peak, and then dropped again.
this shows that higher-order composition made the scheme remember the instances in the training procedure, a phenomenon called overfitting.
therefore, we did not experiment with orders higher than six for ungrouped amino acid composition, and eight for grouped amino acid composition.
localization signals in partial sequences
the sequence signals that guide the sorting of proteins into different subcellular compartments are not well understood.
for a given compartment, more than one targeting signal seem to exist.
apparently, peroxisomal targeting involves at least three different signals, and mitochondrial targeting involves four  <cit> .
although characterized targeting signals are generally short n- or c-terminal peptide motifs, in many cases signals appear to be embedded in the central region of the protein.
this explains why est-peptides, which often lack the n-terminal portion of a protein, are still information-rich enough for inferring subcellular localization, as we have demonstrated here.
in our study, tetra-peptides  yielded the best performance.
this is unexpected for a feature space two orders of magnitude larger  than the sample size .
statistically, most tetra-peptides should be represented by only a few proteins, which should render machine learning rather ineffective.
alternatively, certain tetra-peptides may be over-represented in a given class, either due to a strong location-related signal or an artifact arising from redundancy in the dataset.
to clarify the situation, we scrutinized the tetra-peptides present in our data .
the size of the dataset allows for a total of  <dig>  tetra-peptides.
while the upper limit of distinct tetra-peptides  is  <dig> , our particular dataset contains two third .
about one half  of the occurring motifs are found in single classes and are present in only a few members of a given class .
further, the motifs within a class do not show conserved sequence patterns.
the complementary trend applies to the absence of tetra-peptides: no particular tetra-peptide is absent from only one class.
in sum, there is no notable enrichment of, or bias against, certain tetra-peptide motifs in a class, nor any sign of a data redundancy artifact.
how then does this feature category achieve superior prediction performance?
one possibility is that tetra-peptides bear targeting information in the form of more complex patterns such as nonadjacent correlation of multiple tetra-peptides, which we would not have recognized in our analysis.
CONCLUSIONS
our results described here show that the svm machine learning method, together with sequence features carefully chosen, predicts the subcellular location of est-derived proteins with high accuracy, thus filling the need for a tool tailored to est data.
we implemented testloc as a fully automated pipeline combining est-orf prediction and localization prediction.
this tool opens new avenues for systematic analysis of est data from any eukaryote including challenging photosynthetic taxa such as plants.
authors' contributions
yqs developed and implemented the methods.
gb conceived of the study, participated in its design, and supervised the process.
yqs drafted the manuscript.
both authors read and approved the final manuscript.
supplementary material
additional file 1
influence of sequence similarity on the accuracy of localization prediction by testloc and blast.
click here for file
additional file 2
list of available subcellular localization prediction methods.
click here for file
additional file 3
gene ontology  term of proteins from the plant dataset used in this study.
click here for file
additional file 4
selection of arabidopsis ests corresponding to proteins of known localization.
click here for file
additional file 5
performance of predicting subcellular localization of est-derived proteins, based on each sequence feature .
click here for file
additional file 6
the independent evaluation of svm predictors based on different representations of amino acid composition, measured with sensitivity and positive predictive value.
click here for file
additional file 7
accuracy of estscan for the prediction of start/stop positions of coding regions in est sequences.
click here for file
additional file 8
comparison of prediction performance of available tools and testloc on full-length plant protein sequences.
click here for file
additional file 9
tetra-peptides present in the expanded plant ests data.
click here for file
acknowledgements
we would like to thank geneviève galarneau for programming parts of testloc and evaluating the accuracy of orf prediction, and jean-françois théroux for implementing gapped amino acid composition.
both projects were conducted in the context of a research internship course of the bioinformatics bachelor program at the université de montréal.
we also thank the author of yloc for conducting the prediction on our data.
this work was supported by the canadian institute for advanced research .
yqs received a stipend from the canadian institute for health research  strategic training grant in bioinformatics.

 in eukaryotes, the majority of proteins are encoded in the nuclear genome and translated on ribosomes in the cytosol. proteins are then transported to different subcellular locations, such as the nucleus, mitochondria, chloroplasts, peroxisomes, etc., where they perform their particular roles in various biological processes. knowledge of subcellular location is an important asset in the annotation of newly discovered proteins, as it bears clues about a protein's function. further, knowing the location of proteins and their molecular function allows us to infer where in the cell the corresponding biological process takes place, what the physiological role of this process may be, and how the various processes are spatially integrated. finally, information on the makeup of proteomes from bacteria-derived organelles helps to elucidate the migration of protein-coding genes from the endosymbiont to the host. a variety of experimental approaches are available today for identifying the subcellular localization of proteins, for example, co-expression of fluorescent proteins, immunofluorescence labeling, gene knockout/knockdown, and proteomics techniques such as liquid-chromatography-tandem mass spectrometry. however, for most species, large-scale experimental identification of protein subcellular localization remains too expensive or unfeasible. this has set the stage for bioinformatics approaches to predict localization in silico. can localization of a protein be confidently inferred via finding a homolog of known location by blast? a previous study indicated that localization can be predicted with up to 90% accuracy when blast identity is 50% or more, but that it falls short for more distant sequences. further, this approach ignores established biological knowledge that homologous proteins are not necessarily located in the same cellular compartment. for example, homologous beta oxidation enzymes are targeted to mitochondria in human and peroxisomes in yeast. most importantly, the blast approach fails for divergent and novel proteins as they do not find significant matches in databases. for all these reasons, the bioinformatics community turned to more suited approaches for protein localization prediction. today, more than dedicated tools are available for in silico protein localization prediction based on annotation or solely the sequence of proteins. annotation information includes textual description taken from the swissprot database, the gene ontology database, or pubmed literature. also used for localization prediction is co-occurrence of functional motifs or structural domains in proteins. sequence-based tools recognize specific targeting signals that guide proteins to different cellular compartments. alternatively, proteins are classified according to single amino acid frequency, dipeptide and gapped amino acid pair composition, or physicochemical properties of amino acids. more recently published predictors combine different protein features, or integrate annotation with sequences-based prediction. finally, meta-predictors combine predictions from several heterogeneous tools. two recent studies evaluated the performance of available localization predictors using datasets that contain only sequences not included in, nor similar to, those in the training sets of these predictors. one identified as best performing tools bacello, loctree, protein prowler, targetp, and wolf-psort, and the other evaluated bacello, yloc, muitiloc, and knowpred as best. in general, these tools have lower performance on data from plants compared to non-photosynthetic organisms such as animals and fungi, and this is due to the presence of mitochondria plus chloroplasts in the cell of plants. both organelles descend from endosymbiotic bacteria and have their own machineries for protein import, dna replication, and gene expression. this makes it difficult for the tools to distinguish the proteins from the two organelles. in silico localization prediction tools use full-length protein sequences that are usually inferred from genome sequence. yet, for many eukaryotic groups of interest are only est data available, and it is unlikely that their genomes will be sequenced soon. database, and the taxonomically broad est database ). when attempting to use available localization prediction tools on protein sequences conceptually translated from ests, we realized that prediction accuracy is generally very low. we tested the performance of seven state-of-the-art tools with proteins inferred from plant ests, and the overall accuracies were below 50%. this is not surprising, because these tools have been designed for full-length proteins and not for ests, which often represent only partial coding regions with an average length of ~ residues. further, est-inferred proteins may have an amino acid composition that differs from that of the corresponding full-length proteins. more importantly, est-peptides often lack the n-terminal region of the corresponding proteins, which usually contains the targeting signal. numbers in %. bold numbers are the result of the here described testloc, which is tailored for ests. the results of full-length proteins are compiled in additional file 8 abbreviations: chl, chloroplast; cyt, cytosol; end, endoplasmatic reticulum; ext, extracellular space; mit, mitochondrion; nuc, nucleus; per, peroxisome; pla, plasma membrane; vac, vacuole; sn, sensitivity; ppv, positive predictive value note that all test data have homologs in databases, which in practice is rarely the case; see text finally blast, which we showed above to be unsuited for localization prediction of full-length proteins, is equally unsuited on est-peptide data. even at sequence identity levels above 90%, the class-averaged accuracy for plant ests was below 75%. in practice, the accuracy would be even lower as est projects often discover novel proteins that lack matches in databases. for example, in a large-scale protist est project, more than 60% of ests could not find informative matches. for the ests from such projects, the overall accuracy of localization prediction by blast would thus be less than 30%. we set out to develop a method that is tailored for predicting subcellular localization based on ests. as a test case we used plant data, which, as mentioned above, are more challenging than those from non-photosynthetic taxa. the methodology we developed can be readily applied to ests from any taxonomic groups, and the models we constructed can be easily retrained with sequences from a particular taxon of interest. methods datasets we used two datasets: data from all plants to build and evaluate the localization prediction models, and arabidopsis-only data to evaluate the combined prediction of orfs and localization. collection of protein sequences of experimentally verified subcellular location from swissprot, we collected full-length arabidopsis proteins localized in nine subcellular compartments: chloroplasts, cytosol, endoplasmic reticulum, extracellular space, mitochondria, nucleus, peroxisomes, plasma membrane, and vacuole. protein sequences were selected by the following criteria: 1) they are encoded by the nuclear genome; 2) their subcellular localization is experimentally verified; and 3) the localization annotation is unambiguous. these are strict criteria in order to avoid false positives in the dataset. no bias was detected as to the functional categories of proteins collected this way compared to other studies. arabidopsis ests dataset the ests corresponding to the above arabidopsis proteins were found via a similarity search by blastx in dbest of genbank, using a procedure illustrated in additional file when the aligned region of an est was longer than 90% of its total length and the amino acid identity between the protein and the translated est was >90%, we regarded the pair of est and protein as belonging to the same gene. the selected ests were translated by emboss transeq into amino acid sequences in the frame indicated by the blastx alignment. sequence redundancy within the collected data was reduced by the tool cd-hit so that no pair of sequences shares more than 60% identity. we obtained a dataset of ests. table compiles the number of instances in each subcellular class. plant ests dataset using the same procedure as described above for arabidopsis, we generated est-peptides for all other plants with known localization. these combined with the above described arabidopsis dataset constitute the plant dataset, which was then used to build the expanded plant dataset as specified in the following. expanded plant ests dataset machine learning schemes perform better when trained with larger datasets. but in practice, the size of training data is often limited by their availability. we noticed that in our initial collection of est-peptides, a number of proteins with known subcellular location were absent, because they have no corresponding est sequences in public databases. to construct a training set with optimal coverage, the missing est-peptides were substituted by artificial ones, generated by breaking up full-length proteins into overlapping pieces of ~ residues. in this way, we almost tripled the size of training data. the procedure is described below. the collected full-length protein sequences from plants were processed according to the following rules: when a sequence was shorter than residues, it remained unchanged. when a sequence was to residues long, fragments of length ranging from to residues were generated from both the n-terminus and c-terminus. the range was based on a survey of the length distribution of ests, which revealed a mean of ~ nt with a standard deviation of ~ nt. the n-terminal fragment started within residues from the first methionine, and the c-terminal fragment ended at the last amino acid. this simulated the nature of ests, which usually contain the complete c-terminal, but lack the n-terminal region. when a sequence was longer than residues, an additional central fragment was generated starting anywhere downstream of the first residues, but before the middle position of the original sequence. the fragmented protein sequences were combined with est-derived peptides and clustered by cd-hit using a threshold of 60%. the final dataset contains sequences from species. localization prediction by dedicated tools and blast we collected the best-performing localization prediction tools: targetp, protein prowler, bacello, wolf-psort, yloc, knowpred, and multiloc, and tested their performance on the expanded plant dataset. the results of wolf-posrt and multiloc were obtained from a locally installed version. the prediction of yloc was provided by its author. for the remaining tools, the results were obtained from their corresponding web-server. to assess how blast performs for localization prediction of ests, the est data were blasted against proteins in swissprot, and the localization information of the second match is transferred to the ests. implementation of support vector machine features used to represent the peptide sequences for input of svm physicochemical properties physicochemical properties of amino acids in a protein can be represented by amino acid indices ). the database currently contains features for each amino acid. for each amino acid feature, its value was added up for the whole sequence, and was normalized by the sequence length. subsequently, each est-peptide was converted into a 494-dimension vector. amino acid composition six different types of amino acid composition were calculated. these include the frequency of individual amino acids, di-peptides, tri-peptides, tetra-peptides, penta-peptides, and hexa-peptides in the input sequence. grouped amino acid composition amino acids were grouped according to their properties. the alphabet of amino acids was replaced by an alphabet of size eight or size ten. group c classified amino acids according to their chemical properties, which have shown good performance for localization prediction of full-length proteins. group d classified amino acids according to their structure. after converting est sequences using these new alphabets, the composition of amino acid groups was calculated from 1st to 8th order. gapped amino acid composition this feature represents the frequency of two amino acids separated by x residues, x being the gap length. we experimented with gap lengths from to parameter selection and evaluation of svm predictions in this study, we employed the svm package libsvm, with the radial basis function adopted as kernel function = exp, γ>0), which requires the selection of the kernel parameter γ, and the penalty parameter c. to obtain the optimal parameters and to evaluate the predictions, we performed a 10-fold cross validation scheme for svm parameter selection, followed by a 10-fold independent evaluation to assess the prediction performance. we first randomly divided the whole dataset into ten subsets of equal size. for each iteration of the ten rounds, nine subsets were combined to build svm models, and the remaining subset was used for evaluation. the combined nine subsets were further subdivided into ten groups, whereof nine groups were combined and used to train svm with given values for c and γ, while the remaining group was used to find the optimal combination of the two parameters. finally, we assessed the performance of the selected c and γ values by using the evaluation data subset. performance evaluation we calculated the overall accuracy for all classes combined, as well as the sensitivity, specificity, positive predictive value, and matthews correlation coefficient of each individual class as follows: overall accuracy =∑i=1ntpi∑i=1n* i: the i-th class; n: total number of classes for each class i: sensitivity =tpitpi+fni* specificity =tnitni+fpi* positive predictive value =tpitpi+fpi* matthews correlation coefficient =tpi*tni−fpi*fni*** tp: true positives; fp: false positives; tn: true negatives; fn: false negatives open reading frame prediction for ests prot4est was used for the prediction of open reading frames in est sequences. ests were first aligned with proteins from the ncbi non-redundant sequence database by blastx. the protein-est alignment indicates the correct translation frame. for ests without significant blast matches, we used estscan from within prot4est. estscan predicts orfs based on a hidden markov model by recognizing the species-specific bias in hexanucleotide composition associated with coding and non-coding regions, and generates a matrix to represent this bias. to generate the matrix, we trained estscan with all annotated arabidopsis genomic and mrna data collected from the european molecular biology laboratory nucleotide sequence database. the mrna data were mapped to genomic data in order to find the borders of coding/non-coding regions, which was needed to train the hmm. more details can be found in the publication describing est-scan.  we experimented with different sequence features to represent est-peptides in svm-based prediction of subcellular localization. as detailed in the methods section, features included amino acid composition, grouped amino acid composition reflecting the physicochemical properties, gapped amino acid composition capturing the spatial context, as well as combinations thereof. the performance of the obtained prediction schemes varies considerably as shown below. performance of predictors based on individual features among the sequence features investigated, the best performance was obtained by the svm predictor exploiting the 4th order amino acid composition, with a mcc > for all large classes. a similarly good performance was observed with svm predictors based on the 6th order group-c amino acid composition and the 7th order group-d amino acid composition. unexpectedly, sequence features such as gapped amino acid composition and physicochemical properties represented by aaindices did not yield satisfying results, with mccs for most classes below. numbers are the average of the 10-fold test. numbers in parenthesis are the standard deviation. bold numbers indicate the best values for each metric in each class of the expanded plant dataset. the values for sn and ppv are given in %. mcc, matthews correlation coefficient. for other abbreviations, see footnote to table the three features are 4th order amino acid composition, 6th order group-c amino acid composition and 7th order group-d amino acid composition three classes are underrepresented, because data from these locations are still scarce. we included these classes because in practice, a query sequence could be from any of these locations. were these locations left out, the predictor would inevitably misassign query sequences from these three classes. yet, when these locations are included in the predictor, there is at least the possibility that the query sequence will be correctly assigned, even if the prediction accuracy may be low. eventually, this shortcoming will be alleviated when more data from these locations become available. performance of predictors based on combined features previous studies showed that integration of multiple sequence features improves the performance of localization prediction. we combined all the sequence features described in the methods section first by integrating attributes and second by integrating prediction results. to integrate attributes, the vectors of all sequence features were combined and used as input for svm predictors. this type of integration yielded much lower performance than the best predictor based on a single feature. integration of prediction results from individual features was achieved by a two-layer svm. the first layer consisted of svm predictors based on a single sequence feature, yielding as output the probabilities of the query sequence to belong to each class. the outputs of all first-layer svms were combined and served as input for the second-layer svm. thus, each sequence was converted to a vector of size. the two-layer svm predictor showed similarly good performance as that based on the best single feature. in addition, we integrated the three features that yielded best performance, as well as the predictions from these features. again, these schemes showed a similar mcc as the predictors based on a single feature, but with lower sensitivity and higher positive predictive value for most classes. compared with the prediction from each of the three features individually, their integration often produced a much smaller rate of false positive predictions. orf prediction from ests another challenge in est-peptide-based localization prediction is the correct translation from nucleotide sequence. unlike genomic data, ests often lack start codon and 5'-utr, which otherwise help to detect the correct open reading frame. in addition, ests are products of single-pass reads often containing low quality regions with sequencing errors that further complicate the task. several tools have been developed for orf identification in est sequences. we chose prot4est, which combines similarity-based and machine-learning-based prediction of orfs. the scoring matrix specific to arabidopsis was obtained by training estscan with arabidopsis data, as described in the methods section. the accuracy of orf prediction, evaluated by the percentage of correctly identified start/stop positions of coding regions, was over 70%. implementation of prediction methods and validation with arabidopsis data we built a tool named testloc that combines est translation with localization prediction. testloc has two components. the first takes est nucleotide sequences as input and generates est-peptides via the tool prot4est. the second takes est-peptides as input and outputs the probability that the peptide is targeted to a given subcellular compartment. the current model predicts nine locations: cytosol, endoplasmic reticulum, extracellular space, mitochondria, nucleus, peroxisomes, plasma membrane, vacuole, and chloroplasts. to evaluate the combined prediction of orfs and subcellular localization, we tested the performance of testloc with arabidopsis ests that correspond to proteins of known localization. sequences were represented by three best-performing features: 4th order amino acid composition, 6th order group-c amino acid composition, and 7th order group-d amino acid composition. the resulting predictions showed high mcc values for most classes. testloc can be downloaded and executed locally. the sequence features to use and how to combine them can be chosen by users via editing the configuration file. it should be emphasized that testloc allows users to train new models with their own data, which facilitates the analysis of sequences from other taxonomic groups or individual species. note that testloc is designed for est data, and should not be applied to full-length proteins. discussion effects of various sequence features when experimenting with different kinds of nth order amino acid composition, we observed a common trend: for the training data, the performance improved with increasing order until reaching a peak; for the evaluation data, at first the performance improved with the order, reached a peak, and then dropped again. this shows that higher-order composition made the scheme remember the instances in the training procedure, a phenomenon called overfitting. therefore, we did not experiment with orders higher than six for ungrouped amino acid composition, and eight for grouped amino acid composition. localization signals in partial sequences the sequence signals that guide the sorting of proteins into different subcellular compartments are not well understood. for a given compartment, more than one targeting signal seem to exist. apparently, peroxisomal targeting involves at least three different signals, and mitochondrial targeting involves four. although characterized targeting signals are generally short n- or c-terminal peptide motifs, in many cases signals appear to be embedded in the central region of the protein. this explains why est-peptides, which often lack the n-terminal portion of a protein, are still information-rich enough for inferring subcellular localization, as we have demonstrated here. in our study, tetra-peptides yielded the best performance. this is unexpected for a feature space two orders of magnitude larger than the sample size. statistically, most tetra-peptides should be represented by only a few proteins, which should render machine learning rather ineffective. alternatively, certain tetra-peptides may be over-represented in a given class, either due to a strong location-related signal or an artifact arising from redundancy in the dataset. to clarify the situation, we scrutinized the tetra-peptides present in our data. the size of the dataset allows for a total of tetra-peptides. while the upper limit of distinct tetra-peptides is, our particular dataset contains two third. about one half of the occurring motifs are found in single classes and are present in only a few members of a given class. further, the motifs within a class do not show conserved sequence patterns. the complementary trend applies to the absence of tetra-peptides: no particular tetra-peptide is absent from only one class. in sum, there is no notable enrichment of, or bias against, certain tetra-peptide motifs in a class, nor any sign of a data redundancy artifact. how then does this feature category achieve superior prediction performance? one possibility is that tetra-peptides bear targeting information in the form of more complex patterns such as nonadjacent correlation of multiple tetra-peptides, which we would not have recognized in our analysis.  our results described here show that the svm machine learning method, together with sequence features carefully chosen, predicts the subcellular location of est-derived proteins with high accuracy, thus filling the need for a tool tailored to est data. we implemented testloc as a fully automated pipeline combining est-orf prediction and localization prediction. this tool opens new avenues for systematic analysis of est data from any eukaryote including challenging photosynthetic taxa such as plants. authors' contributions yqs developed and implemented the methods. gb conceived of the study, participated in its design, and supervised the process. yqs drafted the manuscript. both authors read and approved the final manuscript. supplementary material additional file 1 influence of sequence similarity on the accuracy of localization prediction by testloc and blast. click here for file additional file 2 list of available subcellular localization prediction methods. click here for file additional file 3 gene ontology term of proteins from the plant dataset used in this study. click here for file additional file 4 selection of arabidopsis ests corresponding to proteins of known localization. click here for file additional file 5 performance of predicting subcellular localization of est-derived proteins, based on each sequence feature. click here for file additional file 6 the independent evaluation of svm predictors based on different representations of amino acid composition, measured with sensitivity and positive predictive value. click here for file additional file 7 accuracy of estscan for the prediction of start/stop positions of coding regions in est sequences. click here for file additional file 8 comparison of prediction performance of available tools and testloc on full-length plant protein sequences. click here for file additional file 9 tetra-peptides present in the expanded plant ests data. click here for file acknowledgements we would like to thank geneviève galarneau for programming parts of testloc and evaluating the accuracy of orf prediction, and jean-françois théroux for implementing gapped amino acid composition. both projects were conducted in the context of a research internship course of the bioinformatics bachelor program at the université de montréal. we also thank the author of yloc for conducting the prediction on our data. this work was supported by the canadian institute for advanced research. yqs received a stipend from the canadian institute for health research strategic training grant in bioinformatics.
support vector machine is used as computational method and est-peptides are represented by different features such as amino acid composition and physicochemical properties. computational localization prediction is possible based on sequence information alone, and has been successfully applied to proteins from virtually all subcellular compartments and all domains of life. testloc is a localization prediction tool tailored for est data. when testloc was applied to the most challenging test case, it yielded high accuracy. knowing the subcellular location of proteins not only indicates how bio-processes are organized in different cellular compartments, but also contributes to unravelling the function of individual proteins.
 the eukaryotic cell has an intricate architecture with compartments and substructures dedicated to particular biological processes. knowing the subcellular location of proteins not only indicates how bio-processes are organized in different cellular compartments, but also contributes to unravelling the function of individual proteins. computational localization prediction is possible based on sequence information alone, and has been successfully applied to proteins from virtually all subcellular compartments and all domains of life. however, we realized that current prediction tools do not perform well on partial protein sequences such as those inferred from expressed sequence tag data, limiting the exploitation of the large and taxonomically most comprehensive body of sequence information from eukaryotes.  we developed a new predictor, testloc, suited for subcellular localization prediction of proteins based on their partial sequence conceptually translated from ests. support vector machine is used as computational method and est-peptides are represented by different features such as amino acid composition and physicochemical properties. when testloc was applied to the most challenging test case, it yielded high accuracy.  testloc is a localization prediction tool tailored for est data. it provides a variety of models for the users to choose from, and is available for download at http://megasun.bch.umontreal.ca/~shenyq/testloc/testloc.html 