BACKGROUND
despite initial expectations that the elucidation of the complete genome of an organism would enable understanding its biology, the establishment of specific links between genotype and phenotype remains one of the major challenges that biology faces today.
in particular, this applies to complex phenotypes that depend on the effect of many genes.
the identification of phenotype-specific genes or other genomic features opens the way to  formulate testable hypotheses on how the action of these genes may explain the occurrence of that phenotype and  predict the occurrence of that phenotype from the analysis of genomic sequences.
especially, the inference of microbial phenotypes on the basis of genomic features is highly relevant within the context of a growing number of genomic projects.
despite the progress that has been achieved for the investigation of phenotype-specific groups of genes, no practical solution exists for the genome-based prediction of phenotypical properties of prokaryotes.
the association of phenotypic and genotypic traits has been intensively investigated in the field of comparative genomics, mostly by exploiting the fact that organisms that share a particular phenotype are expected to share the set of genes responsible for that trait.
in particular, phylogenetic profiles - presence/absence patterns of a given gene in a set of genomes - have been used to identify the function of uncharacterized proteins based on their co-occurrence with known proteins .
on the other hand, the use of gene co-occurrence patterns is highly affected by missing data resulting from genome misannotation or erroneous assignment of orthology  <cit> .
since the orthology assignment step is a pivotal element of most phylogenomics approaches, the results of such gene-based methods can easily be deteriorated by simple genomic rearrangements such as gene fusion/fission events or domain shuffling  <cit> .
rather than correlating gene occurrence patterns only, several alternative approaches explicitly analyze genotype-phenotype associations by linking genes to a particular phenotype, e.g.
gram stain, oxygen requirement, endospore formation or motility  <cit> .
in order to circumvent the orthology search step and the problem of incomplete presence/absence patterns, most of these approaches map genes and proteins to a more generic level such as clusters of orthologous genes  <cit> , protein domains  <cit>  and metabolic pathways  <cit> .
a natural extension to these approaches is the de novo prediction of an organism's phenotype from the information contained in the genome.
such a predictive approach is likely to be increasingly useful in the context of rapid generation of genomic sequences from a growing number of microorganisms.
for instance, current sequencing techniques allow the determination of genomic sequences of unculturable organisms as well as the rapid generation of sequences from recently isolated strains and even from single cells  <cit> .
in particular, the rapidly growing number and sequencing depth of metagenomes from different natural environments and human body sites will allow the analysis of the genetic potential of the most dominant species without isolation of these organisms  <cit> .
in many of the abovementioned cases, the availability of the genomic sequence will precede the standard phenotypic characterization by means of experimental tests.
to our knowledge, the possibility of directly predicting phenotypic properties from genome content has only been investigated in the context of assessing the predictive power of phenotype-associated genotypic features.
for instance, in kastenmüller et al.
<cit>  genome-specific metabolic pathways have been evaluated for phenotype prediction using an intermediate mapping of proteins to pathways via ec categories together with multi-variate machine learning techniques.
however, as the authors point out, the pathway inference method has several limitations arising from incomplete ec annotations for proteins and the limited metabolic knowledge represented in pathway databases.
furthermore, the approach is limited to pathways that have been detected and described in culturable species.
here we present an approach for the prediction of microbial phenotypes that is entirely based on discriminative learning from protein domain frequencies as obtained from a large number of prokaryotic genomes.
this approach does not require the identification of orthologous genes and fully operates on local genome features, i.e.
the presence or absence of certain pfam  <cit>  domain families.
the latter property in particular allows to make predictions in very early stages of a genome project, just after a sufficient sequencing coverage has been obtained.
provided the sequencing reads are long enough, e.g.
like in recent versions of  <dig> technology and upcoming solexa paired-end modules, this status can already be achieved before assembly of the reads into longer contigs.
thanks to the availability of hmmer <dig> , which achieves the speed of rps-blast  <cit> , and with the availability of the ufo web server  <cit> , which is  <dig> times faster than rps-blast, the detection of protein domains at genomic scale is no longer a computationally expensive task.
RESULTS
our approach for phenotype prediction is based on the complete genomic sequences and the ncbi phenotype annotation of more than  <dig> prokaryotic organisms from  <dig> different phylogenetic groups available at the ncbi web site http://www.ncbi.nlm.nih.gov/.
the predictions were performed by applying a discriminative machine learning technique to the organisms' protein domain profiles, i.e.
frequencies of pfam  <cit>  domain families in their genomes, and the organisms' phenotype annotations .
as we will describe below, our approach has two important outcomes:  the accurate prediction of microbial phenotypes from organism-specific protein domain profiles and  the identification of discriminative, i.e.
phenotype-specific, pfam domain families.
first, we compared our approach to a method that can be used for microbial phenotype prediction and which is based on inferred metabolic pathways  <cit>  .
table  <dig> shows the results of our evaluation in terms of the average values of the product of sensitivity and specificity and the area under roc curve  over  <dig> repetitions of a cross-validation procedure.
the performance values for our domain-based approach are consistently higher than that of the pathway-based method: for both phenotype categories our approach achieves a  <dig> percentage point improvement in terms of the product of sensitivity and specificity.
prediction performance comparison between a pathway-based method  <cit>  and the protein domain profile-based approach.
the first column indicates the phenotype category.
the second and fourth column represent the phenotype prediction performance in terms of the area under roc curve , the third and fifth column denote the prediction performance in terms of the product of sensitivity  and specificity .
for the domain-based method the values denote average  and standard deviation  over  <dig> repetitions of a ten-fold cross-validation with random partitioning.
performance values for the pathway-based methods have been taken from the supplemental material associated with the original work.
afterwards, we extended our evaluation to the whole set of organisms from the ncbi web site .
column  <dig> of table  <dig> shows the phenotype prediction performance of our domain-based approach in terms of the harmonic mean of sensitivity and specificity, which is also known as f1-measure.
the average accuracy over all phenotype categories is high , indicating that the high-dimensional protein domain profile space provides a sufficient separability of examples by the linear classification model.
comparison of phenotype prediction performance for different validation sets according to phylogenetic proximity of organisms .
the first column indicates the phenotype category, the second and third column represent the prediction performance in terms of the harmonic mean of sensitivity and specificity for the validation set without and with using the genus partition, respectively.
values in the fourth column correspond to the difference of prediction performance when the genus-partitioned data set is used.
among the sequenced genomes available at the moment, several entries represent closely related organisms from the same genus or merely different strains from the same species .
to estimate the prediction accuracy in case of the absence of those close relatives, we also evaluated our method on a genus-partitioned data set, i.e.
organisms belonging to genera that have been used for training were not used for performance assessment.
in the evaluation setup at hand this can also be seen as a rather rigorous redundancy reduction on a particular phylogenetic level.
column  <dig> of table  <dig> shows the performance values after genus partitioning.
the average performance decline  is ≈  <dig> percentage points, ranging from ≈  <dig> percentage points for the phenotype category "oxygen requirement" to ≈  <dig> percentage points for the "endospores" category.
to estimate the generalization capability of our approach, we measured the prediction performance on an independent test set of  <dig> proteomes from organisms that have not been used in the validation procedure .
table  <dig> shows the generalization performance in terms of the harmonic mean and other performance indices and figure  <dig> shows the associated roc curves.
generalization performance using an independent test data set .
the first column indicates the phenotype category, the remaining columns represent the prediction accuracy in terms of different performance measures.
here, "aucroc" and "aucprc" correspond to "area under roc curve" and "area under prc curve", respectively.
in principle, our discriminative approach can be used with full genes as genomic features instead of  protein domains.
to evaluate this, we also used clusters of orthologous genes  to construct organism-specific gene frequency profiles .
on average, the validation performance is marginally lower as compared with the protein domain profiles .
the same can be stated for the average generalization performance , although here the sensitivity is slightly higher .
so far, we limited our evaluation to binary cases of phenotypic traits, but in general, phenotype characteristics can have multiple values and may even be continuous.
in contrast to other discriminative machine learning techniques, the method we used provides the possibility of multi-class learning  as a standard feature .
thus, we extended our evaluation to three phenotypes with multiple values: "salinity" , "temperature range"  and "oxygen requirement" including facultative organisms.
since the above mentioned performance indices are not defined for multi-class problems, we measured the generalization performance in terms of the average classification error.
here, the categories "oxygen requirement" and "temperature range" show a good performance with average errors of ≈ 2% and ≈ 11%, respectively.
in contrast, the phenotype category "salinity" yields a classification error of ≈ 33%.
a potential application of the approach that we present here is the prediction of phenotypical traits for organisms that cannot be cultivated but whose genomes can be sequenced using single cell sequencing techniques or may be assembled from metagenomic sequence data because of the organism's abundance.
as a pilot study, we analyzed two microbial genomes obtained from a single cell sequencing approach  <cit> .
our prediction for the two flavobacteria  indicate that both organisms do not form endospores, are gram negative, do not show evidence for motility and a weak evidence for an aerobic lifestyle.
these results are in good agreement with what would be expected from flavobacteria in a marine environment.
discussion
as we showed in the previous section, our discriminative protein domain-based approach to microbial phenotype prediction provides a good validation and generalization performance and outperforms a method based on metabolic pathways  <cit> .
the latter finding can possibly be explained by the fact that the pathway-based method was not intended for the purpose of phenotype prediction, but for the identification of phenotype-associated pathways.
in contrast to our approach, which is designed to identify discriminative features from a large set of protein domains, the restriction to a few detectable metabolic pathways is likely to dismiss many of such discriminative features.
furthermore, the ec-based mapping of proteins to metabolic pathways as described in  <cit>  complicates the practical application for de novo prediction of newly sequenced organisms.
in contrast, our simpler approach only requires the phenotype-specific discriminative weight vectors and the organism's protein domain profile as obtained from a fast domain detection such as ufo  <cit> .
since protein domain sequences are often substantially shorter than the corresponding genes, domain detection can even be performed on short sequence fragments which do not contain full genes.
this also qualifies our approach for phenotype prediction on sequence read data as obtained from high-throughput sequencing, providing a sufficient number of reads can be assigned to a particular species.
however, if the complete protein sequences of an organism are available, our results indicate that clusters of orthologous genes  can be used as well to provide genomic features within our discriminative framework.
this suggests that the choice between full genes and protein domains should depend on the kind of discriminative features one would like to obtain for further analysis .
in principle, our approach could also be used to study the importance of protein domain architecture for microbial genotype-phenotype association, e.g.
by considering particular domain architectures as single features.
however, this requires to deal with several rearrangements such as gene fusion, gene fission and domain shuffling, which lead to a high-dimensional and sparsely occupied feature space that complicates learning.
our results indicate a good generalization performance and thus a real world suitability of our approach.
however, compared to the values obtained during validation, a decreased performance for all phenotype categories except "oxygen requirement" can be observed.
we assume that this results from a combination of sequencing status and annotation quality of the organisms, which we used for the test set: the mostly unfinished genome sequences  may lead to wrongly estimated domain profiles and the sometimes unverified phenotype annotation of these newly sequenced organisms may have been derived merely on the basis of phylogenetic proximity to well-studied organisms.
our genus partition experiment simulated the absence of close species across the training and validation sets and showed that the prediction accuracy decreases differently across the phenotype categories.
these results suggest that certain phenotypic traits can be predicted reliably without the presence of closely related species in the training data, while other phenotype categories lack a representative phylogenetic diversity in current databases and may thus depend on the presence of closely related organisms for accurate prediction.
similarly, our pre-study regarding multi-class phenotypes showed divergent accuracy results for the three tested phenotypes.
in particular, the "salinity" prediction performed poorly, with every third example being wrongly assigned.
this may be explained by the low number of training examples for particular traits  and the difficulty to unambiguously assign one of these phenotype characteristics to an organism.
investigation of discriminative domain families
in order to assess the biological relevance of the discriminative protein domain features, we investigate the  <dig> most discriminative phenotype-specific domain families within a biological context.
for brevity, we restrict the analysis to the two phenotype categories "endospores"  and "motility".
however, the investigation can be performed in a similar fashion for the "gram stain" and "oxygen requirement" phenotypes using the ranked lists of discriminative domain families provided in additional file  <dig>  the features are based on the discriminative model that has been learned from all training organisms using the parameter associated with the best validation performance.
we also discuss the results from the phylogenetic clustering of these families, which may be used to identify functional modules of domains .
the clustering dendrograms associated with all phenotypes can be found in additional file  <dig>
endospores
endospores are dormant resistance forms produced by some gram-positive bacteria by the formation of a thick internal wall that encloses the dna and a part of the cytoplasm.
for reasons of brevity, table  <dig> only shows the  <dig> most discriminative domain families associated with the phenotype category "endospores", a list with the  <dig> most discriminative domains can be found in additional file  <dig>  of the top  <dig>  domains,  <dig>  have the terms "spore" or "sporulation" in their description line.
additional domains are directly involved in sporulation, e.g.
the cell wall hydrolase, the coat f domain, the germination protease, or the yabg peptidase u <dig> domain, which is present in a sporulation-specific protease of several spore coat proteins  <cit> .
other domains such as yabp have been associated with the sporulation process through mutagenesis analyses.
thus, the discriminative domain set for prokaryotes associated with the term "endospore formation" is enriched in domains involved in the sporulation process.
list of the  <dig> most  discriminative domain families associated with the rlsc model for the phenotype category "endospores" .
the first column indicates the rank, the second column shows the discriminative model weight.
the third column denotes the phylogenetic width of a particular domain family, i.e.
the number of taxonomic groups at phylum level in which the family occurs.
the fourth and fifth column correspond to the pfam id and family description associated with a particular domain family.
the table with the  <dig> most positively and negatively discriminative domain families can be found in additional file  <dig>
motility
of the top  <dig>  domains associated with the phenotype "motility" ,  <dig>  are known components of the flagellar machinery, the macromolecular complex responsible for bacterial motility, including spoa and saf domains.
other groups of domains associated with this phenotype are related to the secretory pathway specifically devoted to export components of the flagellum, these include fhipep, flhb, flif, flip, and bacterial export families  <dig> and  <dig>  additional three domains are associated to the process of translation and it could be postulated that they are specific factors for transcribing genes related to the domain families mentioned above.
for instance, sigma- <dig> has recently been found to be involved in the flagellum in e. coli  <cit> .
list of the  <dig> most  discriminative domain families associated with the rlsc model for the phenotype category "motility" .
the first column indicates the rank, the second column shows the discriminative model weight.
the third column denotes the phylogenetic width of a particular domain family.
the fourth and fifth column correspond to the pfam id and family description associated with a particular domain family.
the table with the  <dig> most positively and negatively discriminative domain families can be found in additional file  <dig>
another group of domains that is highly represented in this set is the one related to chemotaxis, the mechanism by which cells can direct their movement in response to certain stimuli.
these include mcp, chew and chec domains, the cache and p <dig> response regulator domains, the signal transducing histidine kinase and its related hamp domain.
it is obvious that all bacteria able of chemotaxis do need a motility system and thus this association is to be expected.
the clustering dendrogram associated with the motility phenotype  is dominated by a cluster that groups together the domain families known to be involved in the flagellar structure and protein families associated with chemotaxis .
this cluster also includes a domain of unknown function , indicating that this duf is highly correlated to flagellar basal-body rod and flagellar hook proteins.
indeed, the pfam annotation of duf <dig> suggests a relation to these families and contains the gene ontology  cellular component category "flagellum".
the duf <dig> domain family has also been identified to be correlated with motility by liu et al.
<cit> .
detecting annotation errors
a possible use of phenotype prediction approaches is the completion and correction of existing annotation.
in order to identify erroneous ncbi annotation associated with the organisms, we compared our prediction results with the phenotype annotation and verified the candidates via literature search.
we found several cases of possible annotation errors for different categories by verifying putatively false predictions on the test set that do not agree with the existing annotation.
for instance, for the "motility" category we identified two organisms  with negative annotation but evidence for being motile  <cit> .
similarly, for the category "gram stain" we found another two possible erroneous annotations  with phenotype annotation in the literature differing from that of the ncbi  <cit> .
finally, we identified two organisms that most likely have been assigned to the wrong respiratory subcategory .
in order to encourage the systematic correction of erroneous existing annotation, we provide phenotype-specific lists with prediction results from our approach for organisms in the independent test in additional file  <dig>
robustness and speed
our validation  data set consists of  <dig>  finished and  <dig>  unfinished genomes, respectively.
the total number of examples used for model training and validation is substantially higher as compared to related work of e.g.
slonim et al.
<cit>   or kastenmüller et al.
<cit>  .
we think that the robustness of models, which is indicated by the good generalization performance, can partly be attributed to the unprecedented size of the data set.
furthermore, the high prediction accurracy on the test set, which contains many yet unfinished genomes, suggests that even in early stages of genome projects reliable statements about an organism's phenotype can be derived from the predicted domain profile.
in contrast to the approach presented in  <cit> , our method does not involve an intermediate mapping of proteins to pathways.
instead, the phenotype prediction is directly performed on the basis of estimated pfam domain profiles.
the computation time that is necessary for inference of the domain profiles depends on the domain detection method and the size of an organism's proteome and ranges from a few seconds to several hours  <cit> .
the actual phenotype prediction can be efficiently performed by means of the dot product of the precomputed domain profile and the discriminative weight vector in domain profile space.
this only requires storage of the 10797-dimensional discriminant and ≈  <dig>  milliseconds for calculation of the prediction score.
CONCLUSIONS
in this work we presented a discriminative machine learning approach that allows to accurately predict the phenotype of a prokaryotic organism directly from its protein domain frequencies.
our results indicate that the domain-based method achieves a better prediction accuracy than the pathway-based approach of kastenmüller et al.
<cit> , while being simpler to implement.
furthermore, we showed that the learned models can be interpreted within a biologically meaningful context in terms of discriminative protein domain families associated with a particular phenotype.
by means of a cluster analysis of these families we also identified phenotype-specific functional modules that contain yet uncharacterized protein domain families  and thus may be used to generate hypotheses about the functions of these domains.
as an alternative to protein domains, we showed that clusters of orthologous genes  may also be used as genomic features for discriminative learning.
similar to our analysis of discriminative protein domains, this might be valuable for interpretation of the resulting phenotype-specific genes.
the incorporation of gene context features such as protein domain architecture is possible with our approach and shall be investigated in the future.
in combination with a fast domain detection method, our approach can be used to rapidly predict the phenotype of an organism solely based on its genomic data.
as we showed in this work, this can be achieved in very early stages even before genome assembly.
in principle, our method can also be used to verify and complete the phenotype annotation in public databases, e.g.
within the ncbi prokaryotic genome project.
future work will include the estimation of the reliability of the existing annotation based on differences between phenotype predictions and literature information.
by this means, we hope to gradually improve the data underlying our method and thus the accurracy of the resulting models.
furthermore, the framework shall be extended to deal with more detailed and diverse phenotype description, e.g.
as provided by the expected growth of metadata associated with sequenced genomes.

 despite initial expectations that the elucidation of the complete genome of an organism would enable understanding its biology, the establishment of specific links between genotype and phenotype remains one of the major challenges that biology faces today. in particular, this applies to complex phenotypes that depend on the effect of many genes. the identification of phenotype-specific genes or other genomic features opens the way to formulate testable hypotheses on how the action of these genes may explain the occurrence of that phenotype and predict the occurrence of that phenotype from the analysis of genomic sequences. especially, the inference of microbial phenotypes on the basis of genomic features is highly relevant within the context of a growing number of genomic projects. despite the progress that has been achieved for the investigation of phenotype-specific groups of genes, no practical solution exists for the genome-based prediction of phenotypical properties of prokaryotes. the association of phenotypic and genotypic traits has been intensively investigated in the field of comparative genomics, mostly by exploiting the fact that organisms that share a particular phenotype are expected to share the set of genes responsible for that trait. in particular, phylogenetic profiles - presence/absence patterns of a given gene in a set of genomes - have been used to identify the function of uncharacterized proteins based on their co-occurrence with known proteins. on the other hand, the use of gene co-occurrence patterns is highly affected by missing data resulting from genome misannotation or erroneous assignment of orthology. since the orthology assignment step is a pivotal element of most phylogenomics approaches, the results of such gene-based methods can easily be deteriorated by simple genomic rearrangements such as gene fusion/fission events or domain shuffling. rather than correlating gene occurrence patterns only, several alternative approaches explicitly analyze genotype-phenotype associations by linking genes to a particular phenotype, e.g. gram stain, oxygen requirement, endospore formation or motility. in order to circumvent the orthology search step and the problem of incomplete presence/absence patterns, most of these approaches map genes and proteins to a more generic level such as clusters of orthologous genes, protein domains and metabolic pathways. a natural extension to these approaches is the de novo prediction of an organism's phenotype from the information contained in the genome. such a predictive approach is likely to be increasingly useful in the context of rapid generation of genomic sequences from a growing number of microorganisms. for instance, current sequencing techniques allow the determination of genomic sequences of unculturable organisms as well as the rapid generation of sequences from recently isolated strains and even from single cells. in particular, the rapidly growing number and sequencing depth of metagenomes from different natural environments and human body sites will allow the analysis of the genetic potential of the most dominant species without isolation of these organisms. in many of the abovementioned cases, the availability of the genomic sequence will precede the standard phenotypic characterization by means of experimental tests. to our knowledge, the possibility of directly predicting phenotypic properties from genome content has only been investigated in the context of assessing the predictive power of phenotype-associated genotypic features. for instance, in kastenmüller et al. genome-specific metabolic pathways have been evaluated for phenotype prediction using an intermediate mapping of proteins to pathways via ec categories together with multi-variate machine learning techniques. however, as the authors point out, the pathway inference method has several limitations arising from incomplete ec annotations for proteins and the limited metabolic knowledge represented in pathway databases. furthermore, the approach is limited to pathways that have been detected and described in culturable species. here we present an approach for the prediction of microbial phenotypes that is entirely based on discriminative learning from protein domain frequencies as obtained from a large number of prokaryotic genomes. this approach does not require the identification of orthologous genes and fully operates on local genome features, i.e. the presence or absence of certain pfam domain families. the latter property in particular allows to make predictions in very early stages of a genome project, just after a sufficient sequencing coverage has been obtained. provided the sequencing reads are long enough, e.g. like in recent versions of technology and upcoming solexa paired-end modules, this status can already be achieved before assembly of the reads into longer contigs. thanks to the availability of hmmer, which achieves the speed of rps-blast, and with the availability of the ufo web server, which is times faster than rps-blast, the detection of protein domains at genomic scale is no longer a computationally expensive task.  our approach for phenotype prediction is based on the complete genomic sequences and the ncbi phenotype annotation of more than prokaryotic organisms from different phylogenetic groups available at the ncbi web site http://www.ncbi.nlm.nih.gov/. the predictions were performed by applying a discriminative machine learning technique to the organisms' protein domain profiles, i.e. frequencies of pfam domain families in their genomes, and the organisms' phenotype annotations. as we will describe below, our approach has two important outcomes: the accurate prediction of microbial phenotypes from organism-specific protein domain profiles and the identification of discriminative, i.e. phenotype-specific, pfam domain families. first, we compared our approach to a method that can be used for microbial phenotype prediction and which is based on inferred metabolic pathways. table shows the results of our evaluation in terms of the average values of the product of sensitivity and specificity and the area under roc curve over repetitions of a cross-validation procedure. the performance values for our domain-based approach are consistently higher than that of the pathway-based method: for both phenotype categories our approach achieves a percentage point improvement in terms of the product of sensitivity and specificity. prediction performance comparison between a pathway-based method and the protein domain profile-based approach. the first column indicates the phenotype category. the second and fourth column represent the phenotype prediction performance in terms of the area under roc curve, the third and fifth column denote the prediction performance in terms of the product of sensitivity and specificity. for the domain-based method the values denote average and standard deviation over repetitions of a ten-fold cross-validation with random partitioning. performance values for the pathway-based methods have been taken from the supplemental material associated with the original work. afterwards, we extended our evaluation to the whole set of organisms from the ncbi web site. column of table shows the phenotype prediction performance of our domain-based approach in terms of the harmonic mean of sensitivity and specificity, which is also known as f1-measure. the average accuracy over all phenotype categories is high, indicating that the high-dimensional protein domain profile space provides a sufficient separability of examples by the linear classification model. comparison of phenotype prediction performance for different validation sets according to phylogenetic proximity of organisms. the first column indicates the phenotype category, the second and third column represent the prediction performance in terms of the harmonic mean of sensitivity and specificity for the validation set without and with using the genus partition, respectively. values in the fourth column correspond to the difference of prediction performance when the genus-partitioned data set is used. among the sequenced genomes available at the moment, several entries represent closely related organisms from the same genus or merely different strains from the same species. to estimate the prediction accuracy in case of the absence of those close relatives, we also evaluated our method on a genus-partitioned data set, i.e. organisms belonging to genera that have been used for training were not used for performance assessment. in the evaluation setup at hand this can also be seen as a rather rigorous redundancy reduction on a particular phylogenetic level. column of table shows the performance values after genus partitioning. the average performance decline is ≈ percentage points, ranging from ≈ percentage points for the phenotype category "oxygen requirement" to ≈ percentage points for the "endospores" category. to estimate the generalization capability of our approach, we measured the prediction performance on an independent test set of proteomes from organisms that have not been used in the validation procedure. table shows the generalization performance in terms of the harmonic mean and other performance indices and figure shows the associated roc curves. generalization performance using an independent test data set. the first column indicates the phenotype category, the remaining columns represent the prediction accuracy in terms of different performance measures. here, "aucroc" and "aucprc" correspond to "area under roc curve" and "area under prc curve", respectively. in principle, our discriminative approach can be used with full genes as genomic features instead of protein domains. to evaluate this, we also used clusters of orthologous genes to construct organism-specific gene frequency profiles. on average, the validation performance is marginally lower as compared with the protein domain profiles. the same can be stated for the average generalization performance, although here the sensitivity is slightly higher. so far, we limited our evaluation to binary cases of phenotypic traits, but in general, phenotype characteristics can have multiple values and may even be continuous. in contrast to other discriminative machine learning techniques, the method we used provides the possibility of multi-class learning as a standard feature. thus, we extended our evaluation to three phenotypes with multiple values: "salinity", "temperature range" and "oxygen requirement" including facultative organisms. since the above mentioned performance indices are not defined for multi-class problems, we measured the generalization performance in terms of the average classification error. here, the categories "oxygen requirement" and "temperature range" show a good performance with average errors of ≈ 2% and ≈ 11%, respectively. in contrast, the phenotype category "salinity" yields a classification error of ≈ 33%. a potential application of the approach that we present here is the prediction of phenotypical traits for organisms that cannot be cultivated but whose genomes can be sequenced using single cell sequencing techniques or may be assembled from metagenomic sequence data because of the organism's abundance. as a pilot study, we analyzed two microbial genomes obtained from a single cell sequencing approach. our prediction for the two flavobacteria indicate that both organisms do not form endospores, are gram negative, do not show evidence for motility and a weak evidence for an aerobic lifestyle. these results are in good agreement with what would be expected from flavobacteria in a marine environment. discussion as we showed in the previous section, our discriminative protein domain-based approach to microbial phenotype prediction provides a good validation and generalization performance and outperforms a method based on metabolic pathways. the latter finding can possibly be explained by the fact that the pathway-based method was not intended for the purpose of phenotype prediction, but for the identification of phenotype-associated pathways. in contrast to our approach, which is designed to identify discriminative features from a large set of protein domains, the restriction to a few detectable metabolic pathways is likely to dismiss many of such discriminative features. furthermore, the ec-based mapping of proteins to metabolic pathways as described in complicates the practical application for de novo prediction of newly sequenced organisms. in contrast, our simpler approach only requires the phenotype-specific discriminative weight vectors and the organism's protein domain profile as obtained from a fast domain detection such as ufo. since protein domain sequences are often substantially shorter than the corresponding genes, domain detection can even be performed on short sequence fragments which do not contain full genes. this also qualifies our approach for phenotype prediction on sequence read data as obtained from high-throughput sequencing, providing a sufficient number of reads can be assigned to a particular species. however, if the complete protein sequences of an organism are available, our results indicate that clusters of orthologous genes can be used as well to provide genomic features within our discriminative framework. this suggests that the choice between full genes and protein domains should depend on the kind of discriminative features one would like to obtain for further analysis. in principle, our approach could also be used to study the importance of protein domain architecture for microbial genotype-phenotype association, e.g. by considering particular domain architectures as single features. however, this requires to deal with several rearrangements such as gene fusion, gene fission and domain shuffling, which lead to a high-dimensional and sparsely occupied feature space that complicates learning. our results indicate a good generalization performance and thus a real world suitability of our approach. however, compared to the values obtained during validation, a decreased performance for all phenotype categories except "oxygen requirement" can be observed. we assume that this results from a combination of sequencing status and annotation quality of the organisms, which we used for the test set: the mostly unfinished genome sequences may lead to wrongly estimated domain profiles and the sometimes unverified phenotype annotation of these newly sequenced organisms may have been derived merely on the basis of phylogenetic proximity to well-studied organisms. our genus partition experiment simulated the absence of close species across the training and validation sets and showed that the prediction accuracy decreases differently across the phenotype categories. these results suggest that certain phenotypic traits can be predicted reliably without the presence of closely related species in the training data, while other phenotype categories lack a representative phylogenetic diversity in current databases and may thus depend on the presence of closely related organisms for accurate prediction. similarly, our pre-study regarding multi-class phenotypes showed divergent accuracy results for the three tested phenotypes. in particular, the "salinity" prediction performed poorly, with every third example being wrongly assigned. this may be explained by the low number of training examples for particular traits and the difficulty to unambiguously assign one of these phenotype characteristics to an organism. investigation of discriminative domain families in order to assess the biological relevance of the discriminative protein domain features, we investigate the most discriminative phenotype-specific domain families within a biological context. for brevity, we restrict the analysis to the two phenotype categories "endospores" and "motility". however, the investigation can be performed in a similar fashion for the "gram stain" and "oxygen requirement" phenotypes using the ranked lists of discriminative domain families provided in additional file the features are based on the discriminative model that has been learned from all training organisms using the parameter associated with the best validation performance. we also discuss the results from the phylogenetic clustering of these families, which may be used to identify functional modules of domains. the clustering dendrograms associated with all phenotypes can be found in additional file endospores endospores are dormant resistance forms produced by some gram-positive bacteria by the formation of a thick internal wall that encloses the dna and a part of the cytoplasm. for reasons of brevity, table only shows the most discriminative domain families associated with the phenotype category "endospores", a list with the most discriminative domains can be found in additional file of the top domains, have the terms "spore" or "sporulation" in their description line. additional domains are directly involved in sporulation, e.g. the cell wall hydrolase, the coat f domain, the germination protease, or the yabg peptidase u domain, which is present in a sporulation-specific protease of several spore coat proteins. other domains such as yabp have been associated with the sporulation process through mutagenesis analyses. thus, the discriminative domain set for prokaryotes associated with the term "endospore formation" is enriched in domains involved in the sporulation process. list of the most discriminative domain families associated with the rlsc model for the phenotype category "endospores". the first column indicates the rank, the second column shows the discriminative model weight. the third column denotes the phylogenetic width of a particular domain family, i.e. the number of taxonomic groups at phylum level in which the family occurs. the fourth and fifth column correspond to the pfam id and family description associated with a particular domain family. the table with the most positively and negatively discriminative domain families can be found in additional file motility of the top domains associated with the phenotype "motility", are known components of the flagellar machinery, the macromolecular complex responsible for bacterial motility, including spoa and saf domains. other groups of domains associated with this phenotype are related to the secretory pathway specifically devoted to export components of the flagellum, these include fhipep, flhb, flif, flip, and bacterial export families and additional three domains are associated to the process of translation and it could be postulated that they are specific factors for transcribing genes related to the domain families mentioned above. for instance, sigma- has recently been found to be involved in the flagellum in e. coli. list of the most discriminative domain families associated with the rlsc model for the phenotype category "motility". the first column indicates the rank, the second column shows the discriminative model weight. the third column denotes the phylogenetic width of a particular domain family. the fourth and fifth column correspond to the pfam id and family description associated with a particular domain family. the table with the most positively and negatively discriminative domain families can be found in additional file another group of domains that is highly represented in this set is the one related to chemotaxis, the mechanism by which cells can direct their movement in response to certain stimuli. these include mcp, chew and chec domains, the cache and p response regulator domains, the signal transducing histidine kinase and its related hamp domain. it is obvious that all bacteria able of chemotaxis do need a motility system and thus this association is to be expected. the clustering dendrogram associated with the motility phenotype is dominated by a cluster that groups together the domain families known to be involved in the flagellar structure and protein families associated with chemotaxis. this cluster also includes a domain of unknown function, indicating that this duf is highly correlated to flagellar basal-body rod and flagellar hook proteins. indeed, the pfam annotation of duf suggests a relation to these families and contains the gene ontology cellular component category "flagellum". the duf domain family has also been identified to be correlated with motility by liu et al.. detecting annotation errors a possible use of phenotype prediction approaches is the completion and correction of existing annotation. in order to identify erroneous ncbi annotation associated with the organisms, we compared our prediction results with the phenotype annotation and verified the candidates via literature search. we found several cases of possible annotation errors for different categories by verifying putatively false predictions on the test set that do not agree with the existing annotation. for instance, for the "motility" category we identified two organisms with negative annotation but evidence for being motile. similarly, for the category "gram stain" we found another two possible erroneous annotations with phenotype annotation in the literature differing from that of the ncbi. finally, we identified two organisms that most likely have been assigned to the wrong respiratory subcategory. in order to encourage the systematic correction of erroneous existing annotation, we provide phenotype-specific lists with prediction results from our approach for organisms in the independent test in additional file robustness and speed our validation data set consists of finished and unfinished genomes, respectively. the total number of examples used for model training and validation is substantially higher as compared to related work of e.g. slonim et al. or kastenmüller et al.. we think that the robustness of models, which is indicated by the good generalization performance, can partly be attributed to the unprecedented size of the data set. furthermore, the high prediction accurracy on the test set, which contains many yet unfinished genomes, suggests that even in early stages of genome projects reliable statements about an organism's phenotype can be derived from the predicted domain profile. in contrast to the approach presented in, our method does not involve an intermediate mapping of proteins to pathways. instead, the phenotype prediction is directly performed on the basis of estimated pfam domain profiles. the computation time that is necessary for inference of the domain profiles depends on the domain detection method and the size of an organism's proteome and ranges from a few seconds to several hours. the actual phenotype prediction can be efficiently performed by means of the dot product of the precomputed domain profile and the discriminative weight vector in domain profile space. this only requires storage of the 10797-dimensional discriminant and ≈ milliseconds for calculation of the prediction score.  in this work we presented a discriminative machine learning approach that allows to accurately predict the phenotype of a prokaryotic organism directly from its protein domain frequencies. our results indicate that the domain-based method achieves a better prediction accuracy than the pathway-based approach of kastenmüller et al., while being simpler to implement. furthermore, we showed that the learned models can be interpreted within a biologically meaningful context in terms of discriminative protein domain families associated with a particular phenotype. by means of a cluster analysis of these families we also identified phenotype-specific functional modules that contain yet uncharacterized protein domain families and thus may be used to generate hypotheses about the functions of these domains. as an alternative to protein domains, we showed that clusters of orthologous genes may also be used as genomic features for discriminative learning. similar to our analysis of discriminative protein domains, this might be valuable for interpretation of the resulting phenotype-specific genes. the incorporation of gene context features such as protein domain architecture is possible with our approach and shall be investigated in the future. in combination with a fast domain detection method, our approach can be used to rapidly predict the phenotype of an organism solely based on its genomic data. as we showed in this work, this can be achieved in very early stages even before genome assembly. in principle, our method can also be used to verify and complete the phenotype annotation in public databases, e.g. within the ncbi prokaryotic genome project. future work will include the estimation of the reliability of the existing annotation based on differences between phenotype predictions and literature information. by this means, we hope to gradually improve the data underlying our method and thus the accurracy of the resulting models. furthermore, the framework shall be extended to deal with more detailed and diverse phenotype description, e.g. as provided by the expected growth of metadata associated with sequenced genomes. 
we present a novel approach to predict the phenotype of prokaryotes directly from their protein domain frequencies. fast and accurate prediction of microbial phenotypes based on genomic protein domain content is feasible and has the potential to provide novel biological insights. first results of a systematic check for annotation errors indicate that our approach may also be applied to semi-automatic correction and completion of the existing phenotype annotation. accurately predicting microbial phenotypes solely based on genomic features will allow us to infer relevant phenotypic characteristics when the availability of a genome sequence precedes experimental characterization, a scenario that is favored by the advent of novel high-throughput and single cell sequencing techniques. moreover, the set of discriminative domains provides biological insight into the underlying phenotype-genotype relationship and enables deriving hypotheses on the possible functions of uncharacterized domains. our discriminative machine learning approach provides high prediction accuracy of relevant phenotypes such as motility, oxygen requirement or spore formation.
 establishing the relationship between an organism's genome sequence and its phenotype is a fundamental challenge that remains largely unsolved. accurately predicting microbial phenotypes solely based on genomic features will allow us to infer relevant phenotypic characteristics when the availability of a genome sequence precedes experimental characterization, a scenario that is favored by the advent of novel high-throughput and single cell sequencing techniques.  we present a novel approach to predict the phenotype of prokaryotes directly from their protein domain frequencies. our discriminative machine learning approach provides high prediction accuracy of relevant phenotypes such as motility, oxygen requirement or spore formation. moreover, the set of discriminative domains provides biological insight into the underlying phenotype-genotype relationship and enables deriving hypotheses on the possible functions of uncharacterized domains.  fast and accurate prediction of microbial phenotypes based on genomic protein domain content is feasible and has the potential to provide novel biological insights. first results of a systematic check for annotation errors indicate that our approach may also be applied to semi-automatic correction and completion of the existing phenotype annotation. 